The cost of buying a home will be a major policy focus for the Turnbull government over its three-year term.
Treasurer Scott Morrison says he wants to work with the states to reform planning laws and increase housing supply to help push house prices down.
He admits it is getting harder to get into the market, especially for first-time home buyers.
“The housing market is getting away from people, particularly if you're in the eastern states,” he said during a lunchtime address to the Urban Development Institute in Sydney.
Prime Minister Malcolm Turnbull agrees that a greater supply would help make housing more affordable.
“It's very clear what the cause is, we simply haven't been building enough dwellings,” Mr Turnbull told reporters in Sydney.
The Treasurer says prices in some parts of Sydney are now nine times what people are earning.
He has also warned the sky-rocketing cost of homes is putting more pressure on the private rental market and on social housing.
Figures from the Australian Bureau of Statistics show that while house prices in Sydney have jumped more than 40 per cent over the past three years, that hasn't been the case across all capital cities.
In Perth and Darwin, prices have fallen by 2 per cent and 5 per cent respectively.
“There is no one single housing market in Australia, it's different all across the country,” Mr Morrison told 2GB radio in Sydney.
Another issue over the coming years is a foreseeable apartment glut in the eastern capitals of Brisbane, Melbourne and Sydney.
Director of Deloitte Access Economics, Chris Richardson, told SBS that between now and the end of 2018, too many apartments will be built for the number of likely buyers.
"Australians know that housing affordability is a problem."
“For inner-city apartments in the big state capitals, prices by the end of 2018 will likely be 15 per cent lower than they are today, so there is some relief coming but it's very specific,” he said.
He said while moves by the government to increase housing supply were welcome, it isn't the key driver around housing affordability.
“The biggest single thing that could affect affordability would be interest rates finally going up for a while instead of down. That's not on the cards in the next year or so but eventually rates will be higher and housing will be more affordable because of that,” he said.
Labor has ridiculed the government’s decision to address the issue of housing affordability now instead of before the July election.
“They want to pretend they're a hero and they say they discovered housing affordability is a problem. Australians know that housing affordability is a problem,” Bill Shorten told reporters in Perth.
The opposition maintains its policy to reign in negative gearing tax concessions would even-up the playing field.
But the treasurer says Labor's policy is ill-considered.
“You don't address housing affordability by crashing the housing market or your economy,” Mr Morrison said.
He will meet with his state and territory counterparts to discuss the issue at a meeting in December.