Kwasi Kwarteng has said massive tax cuts aimed at boosting economic growth are fair for all despite the highest earners gaining the most.
The chancellor scrapped the top rate of income tax as part of the biggest package of tax cuts in 50 years.
Labour and some Tory MPs have said it was wrong to cut taxes for the wealthy during a cost-of-living crisis.
But Mr Kwarteng said he was "being fair" by reducing taxes right across the income bracket.
The package of measures, which has been dubbed a mini-budget, will be paid for by a sharp rise in government borrowing amounting to tens of billions of pounds.
Paul Johnson, director of the independent Institute for Fiscal Studies, said the plans were a "big gamble", with money being pumped into the economy when inflation remains high.
There was an immediate reaction in financial markets, as the pound sunk and UK stocks fell.
In an interview with BBC News political editor Chris Mason, the chancellor said: "I don't think it's a gamble at all.
"What was a gamble, in my view, was sticking to the course we are on."
Mr Kwarteng insisted not cutting taxes and continuing to follow the path of the previous government - led by former Prime Minister Boris Johnson - was more risky.
"So what we had to do was have a reboot, a rethink," Mr Kwarteng said.
He said his mini-budget was not an admission of failure, as Labour suggested in response to his statement, and a recession forecasted by the Bank of England was "not inevitable"
But, he said, "we also recognise we could do things better".
He said his moves to reduce income tax, scrap a planned rise in National Insurance and support households with energy bills would "help the most vulnerable people in society get through a difficult time".
Asked whether the UK economy was now in recession, Mr Kwarteng said that while, "technically, the Bank of England said that there was a recession", he thought it "would be shallow" and he hoped "we would rebound and grow".
When pressed, the chancellor said he did not acknowledge that the UK was in recession and that one was not inevitable.
The Institute of Fiscal Studies said the richest 10% of households would gain the most from Mr Kwarteng's measures, which undo the tax rises introduced by former chancellor Rishi Sunak, who left office in July.
Labour's shadow business secretary Seema Malhotra told BBC Two's Newsnight: "What was really striking today in Parliament was the admission, effectively, of 12 years of failure of what the Conservatives have been doing.
"And Kwasi Kwarteng and his colleagues failed to connect the fact that we've got high taxes because we've had low growth."
She added: "If you're going to have a plan for growth, it's got to be a proper plan. Even today wasn't a proper plan. Today was a set of tax cuts that are supporting the wealthiest in our society."
Torsten Bell, the chief executive of the Resolution Foundation think tank, said those earning £1m annually will get a £55,000 tax cut next year.
Conservative former cabinet minister Julian Smith said the chancellor's decision to hand a "huge" tax cut to the wealthy at a time of national crisis was "wrong".
When Mr Smith's concerns were put to the chancellor, he said the former chief whip "knows about party loyalty" and "lots of people feel we've got to get Britain moving".
Getting the economy moving will, according to Mr Kwarteng, require doing things differently, what he called "a new approach for a new era, focused on growth".
That new approach will be funded by borrowing, which economists think could climb to £120bn within three years.
In Parliament, John Glen, a former junior finance minister, pointedly asked Mr Kwarteng about market reaction, saying "there is a clear concern" over the level of borrowing needed to afford these tax cuts.
Prime Minister Liz Truss, who took office this month, has said she was prepared to make "unpopular decisions" to spur economic growth and support those struggling with the rising cost of living.
She won the Tory leadership election on a platform of upending what she has called the economic "orthodoxy" of the Treasury and reversing tax hikes introduced by her predecessor.
Now Mr Kwarteng is putting her economic agenda to the test.