‘You can’t call me that!” Sayeeda Warsi splutters, and I can’t help but smile. After all, this is the 45-year-old Yorkshirewoman and Britain’s first Muslim woman cabinet member, who prides herself on plain speaking. So far in our interview she has admitted to having “humdingers of rows” while in government, and blasted “lazy policymaking” on counter-terrorism. But she is still aghast when I describe her as “going rogue” since she resigned from the government over Gaza policy in 2014.

Let’s just say Lady Warsi – who was put on an Islamic State “kill list” last year – has been quite outspoken in the intervening years, slamming Zac Goldsmith’s divisive London mayoral campaign, condemning David Cameron for “demonising Muslims” and branding the campaign to leave the EU xenophobic. Yet, given the chance to write a warts-and-all-memoir of her time in cabinet, she has instead produced a densely argued book on British Muslims and how they are positioned as the latest “enemy within”.

She says it was the prospect of becoming a grandmother that gave the subject a new urgency. Despite her position in the establishment – the news made her think: “Where are my grandkids going to call home? What world will they grow up in? This is about more than just me. This nonsense we have to deal with on a day-to-day basis, that,” she deepens her voice, “the Muslims are taking over! They are all extremists and terrorists!” she breaks off. “Let’s get some reality, evidence and fact into the debate.”

She also wants to reassure a younger, post-9/11 generation that there wasn’t always a “ginormous LED spotlight” on British Muslims. And that with the right solutions and a more nuanced debate, there is every reason for optimism.

It can feel that no aspect of British Muslim life has not been scrutinised through the lens of counter-terrorism. Mosques and religious teachings are often foremost in the spotlight, but British Muslim family life is the home front in an ongoing attempt to find potential causes of radicalisation.

From the neighbourhoods Muslims live in to who they marry, how they divorce to what they teach their children, what religious education their children have, all the way down to what language they speak to them – all have been debated and dissected.

Warsi has always been willing to confront issues within British Muslim communities – she was pelted with eggs by protestors demanding Sharia law in 2009. She also confronts her own failings – notably using homophobic campaign material in 2005 (she calls it “toe-curlingly embarrassing and deeply offensive”). But she is aghast at accusations there are swaths of the community who condone terrorism. As she points out drily, if the more than 3 million British Muslims could really not be trusted: “We would have killed ourselves and everyone else by now.”

She charts how we arrived at a place where a “community that came to work and go back became a community that worked and settled, and is now viewed as neither settled nor belonging”.

She argues that government policy towards British Muslim communities is today entirely focused through the prism of the “one tenth of 1% [who] have ever been involved in any form of activity relating to terrorism”, while Muslim organisations are under scrutiny as never before, and Muslims are vilified in the media.

Her starting point is her own upbringing in Dewsbury, West Yorkshire. After hearing her complain about the tea we are drinking not being Yorkshire, it’s no surprise that she writes fondly of her “childhood in Savile Town in the late 70s and early 80s” with its “brass bands … hopscotch and British bulldog”. Her parents arrived there from Punjab, Pakistan, in the 1960s and her father worked as a bus driver then a taxi driver before making his fortune with a bed manufacturing firm. She talks with nostalgia of her “tough working-class home”, and how, despite it being a time when “Paki” was the favoured insult, her parents deliberately chose a mixed community to expose their children “to a plural and diverse ethnic and religious upbringing”.

An unwillingness to integrate is one accusation often flung at British Muslim families. Yet Warsi insists the reality is more complicated.

“When we were growing up … the neighbours on both sides were English, so you didn’t need to worry about integration.” Now, she says, with Asian communities being so much larger, “integration is a middle-class pastime. If you have a nice house, in a nice area, where people from all different backgrounds send their kids to the same posh schools, you can have nice dinner parties. But if you are at the bottom of the pile – whether you are white, black, or anything else – then integration is not the biggest thing on your mind. If that’s the only house you can afford to live in, or the one you have been allocated, and those are the schools you have been told to go to, integration is something you have to seek, and it’s not your biggest priority.”

Nor was opposition to it ever just a one-way street. “White flight was a big part [of it] when I was growing up. When I first moved to a more white area in the early 90s, I would get letters saying ‘Pakis get out’. Our house would get egged. It wasn’t like people embraced us with open arms.”

Yet integration is now considered as something “those who are not Anglo-Saxon” must do, rather than about “how all of us get on, irrespective of who we are and where we came from”.

As with so many social issues British Muslims face, lack of integration is often cited as a possible cause of radicalisation. The recent Casey review on this was part of the government’s counter-terrorism strategy. “It started out as a report on integration and ended up as a report on Muslims,” Warsi says. This meant that any opportunity to tackle problems facing deprived white communities who don’t feel they have a stake in society, was lost. “I think Louise [Casey] would tell you there was a lot of internal politics about what could be published.”

Yet it’s not just where Muslims live that has been challenged. In one counter-terrorism speech, Cameron suggested that Muslim mothers who don’t speak English could be susceptible to extremism. Warsi bristles: “My mum’s English wasn’t great, but she stood over us when we did our homework. She produced a cabinet minister, a lawyer, a chartered accountant, a pharmacist – she didn’t produce a terrorist.”

Warsi herself agrees those seeking spousal visas must learn English, so I wonder what makes her so uncomfortable? “I am completely with Cameron on that,” she says, “but it’s not counter-terrorism. It’s so they can go to hospital, open a bank account and live a complete life.”

Madrasas – Islamic sunday schools – are another example, she says. In another counter-terrorism speech, Cameron suggested they could fill children’s heads with “poison”. “I came out of those madrasas,” Warsi says drily. “Dewsbury might have produced bombers, but it also produced the first Muslim cabinet minister.”

It’s a similar issue with the current government inquiry into Sharia councils. The councils, which facilitate Islamic divorces for Muslim women who need a religious scholar to end their marriage if their husbands don’t consent, are particularly vital to Muslim women who only have a religious rather than a civil marriage. Now a government inquiry is investigating whether the councils discriminate against women. It’s an inequality issue being framed as a counter-terrorism one, says Warsi.

“Is making sure Muslim women have the same opportunities as Muslim men a counter-terrorism thing? No.”

Here, too, she can point to her own experiences growing up as one of five sisters in a male-dominated, conservative society. “It shaped my personality, my drive, and indeed the chip on my shoulder,” she writes. Girls, she continues, were seen as a liability among the Asian community in Dewsbury.

But, she says, it worked in her favour, because it meant her parents became determined they would all be highly educated professionals. “Mum decided – in a very traditional Asian way: ‘If they hit hard times, do I want them behind a checkout or in a nice office?’ The second part was: ‘I have five girls – who will marry them if they are uneducated country bumpkins?’”

As the sisters left for university, there was disapproval within the community – and Warsi admits this freedom came with caveats. “As my eldest sister said, ‘I knew if I screwed up, it wasn’t going to happen for the rest of you.’

“The more the elder ones behaved, the more [freedom] the younger ones could have. But we didn’t resent it.”

It also meant that, while Warsi and her older sister had arranged marriages, the younger sisters chose their own partners. Warsi married at 19. “There were girls getting married at 15 among my friends; girls weren’t allowed to complete their education. I was at university, had passed my driving test, had a car and was on a trajectory to be a lawyer – life was good.”

In 2005, when she became an MP in Dewsbury, she encountered the same attitudes towards women. “Some of the language I had to listen to [from men]: ‘Our women will be out of control if someone like her gets elected,’” she says, laughing, “as though I would pipe them out of the town!”

Warsi’s first marriage, during which she had a daughter, Aamna, lasted 17 years. In 2009, she married Iftikhar Azam, a father of four. This match, too, she says, was “semi-arranged”, and “the best decision I ever made”.

A few years on, she says attitudes in the Muslim community towards female politicians have softened. “The concept of Muslim women in politics and leadership positions has changed dramatically. Girls come up to me and say, ‘My dad says I want you to be like Sayeeda Warsi!’”

Yet, for all the advances typified by her career, the rise in Islamophobia has continued and Warsi is in no doubt things have got much worse. “In 2011, I said Islamophobia had passed the ‘dinner table test’ [to become socially acceptable] and you would think I had thrown a grenade into middle Britain,” she recalls, smiling. “In 2017, people wouldn’t bat an eyelid at that phrase.”

She has certainly not remained unscathed by anti-Muslim feeling. When Warsi was made a peer in 2007, Conservative Home ran an article that said this sent “the wrong signal at a time when Britain is fighting a global war against Islamic terrorism and extremism”.

And, she writes, after a reshuffle saw her move to minister of faith at the Department for Communities and Local Government in 2012 (seen as a demotion) her special adviser was asked to “effectively spy” on her by someone employed by the Conservative party.

“Being a Muslim in public life has been brutal,” she says simply.

The atmosphere of suspicion towards British Muslims is having a deeply corrosive effect outside politics too, she says. “Muslims who engage with politics or any other … institutions are to be viewed as suspicious,” she writes, “and Muslims who don’t engage are to be treated as suspicious for being separatist.”

The government is so paranoid about working with Muslim groups that might hold “non-violent extremist” views (defined worryingly loosely as “opposition to British values”) that it has stopped engaging with increasing numbers of Muslim organisations.

The Prevent strategy is another area of concern. Teachers and even nursery staff have a statutory duty to spot children who might be vulnerable to radicalisation and, if necessary, refer them to the government’s anti-radicalisation programme. In her book, Warsi cites well-known cases of Muslim children questioned about having a toy gun, or using the word L’ecoterrorisme in a French class and says distrust of the programme means it must be reassessed. It’s not a new argument – but it is one that is more powerful coming from someone who has been at the heart of government.

“Prevent should be the parent’s friend,” she says. “The place where, if you are worried about your kids, you can go and get help and support … it’s not. It started off well with noble intentions … but it has to be something that has complete buy-in from the communities in which it is working.”

Warsi says she is at odds with the Conservative party’s attitude towards British Muslims, not because she has changed her views, but because the party has “lurched” in the past decade. Yet she insists this is no time to be pessimistic. She even manages to spin outright Islamophobia as a reason to be cheerful. “When people are feeling despondent, I say it’s better now because things are so easy to see,” she says.

In her book, she writes: “In 2012, the Conservative party decided that Muslims were unlikely to vote for them … the battle to engage the Muslim community on ‘normal’ issues or indeed any issue other than terrorism was a daily grind.” So, I wonder, what could make the government have a change of heart? She looks at me, in surprise. “A better country,” she says firmly. “Theresa May has a unique opportunity to get this right. When the fog of fascism seems to come in from the east and west, we as a nation have to determine who we want to be.”

This interview was carried out before Wednesday’s attack in Westminster.

The Enemy Within by Sayeeda Warsi (Penguin, £20). To order a copy for £15 go to bookshop.theguardian.com or call 0330 333 6846. Free UK p&p over £10, online orders only. Phone orders min. p&p of £1.99.