Sir Jeffrey Donaldson, the new leader of Northern Ireland’s Democratic Unionist party, said on Thursday that he will push to remove the controversial post-Brexit customs border in the Irish Sea within “weeks”.

Donaldson, the veteran Westminster MP and third person to lead the DUP in less than two months, used his inaugural speech as leader of the party to strike a combative tone on the Northern Ireland Protocol — part of the UK’s Brexit treaty which averts the need for a hard border on the island of Ireland.

Diplomats in Brussels and London have been working to defuse tensions over the protocol which keeps Northern Ireland in the EU’s single market for goods and introduces checks on goods moving between Great Britain and the region. On Wednesday, the UK and EU agreed a three month extension to grace periods so that chilled meat can be imported without checks.

While Donaldson’s predecessor, Edwin Poots, said he would take a gradual approach, the DUP’s new leader said his goal was to remove the border “in the weeks ahead”.

Asked whether he would pull down the region’s power sharing government at Stormont if he is unsuccessful, Donaldson said: “I wouldn’t use those words. I’m not here this morning saying that’s what I want to do, far from it . . . I don’t want to be in that space, but I have to be realistic.”

Unionists argue that the protocol denies them their right of free access to goods from the rest of the UK and is an assault on their British identity.

Donaldson said the border had already caused “instability” and that the UK had a responsibility to work towards restoring equilibrium in the region, as did Ireland which can’t “have its cake and eat it” by expecting good relations with Belfast while supporting a policy that hurts Northern Ireland.

The MP did not set out an alternative to the current arrangements, beyond discontinuing customs checks, but offered the example of “one of my elderly constituents in Lisburn planting some seeds in her garden” as proof that British goods coming into Northern Ireland were for domestic use and should not be subject to any checks.

“It’s not about the integrity of the EU single market, it’s about punishing the UK because they dared to leave the EU, it’s about using Northern Ireland as a stick to beat the UK with, and that has to stop,” he said.

The new DUP leader said Wednesday’s High Court decision to throw out unionists’ challenge to the protocol’s legality “could not have been clearer” that the protocol “undermined the Act of Union, the very basis of our relationship with the UK”. Unionists are set to challenge the ruling, which found among other things that the 2018 Withdrawal Act overrode the 1800 Act of Union.

Donaldson also warned that next May’s Assembly elections would be treated by some as a “referendum on a border poll that would plunge Northern Ireland into division and instability”.

Nationalist party Sinn Féin, the second largest group in Stormont and most popular party according to polls, is pushing for a vote on a united Ireland. Donaldson said unionists should come together to “resist this threat”.

He acknowledged recent months, when the DUP blazed through two leaders amid bitter infighting, had “not been the proudest in the history of our party” and have “strained the patience, not just of the public but of our own supporters as well”. “I want to apologise to the public for that.”