The three are among a new generation in the city, which returned to Chinese rule in 1997, demanding greater self determination, at least six of whom won seats in its 70-member legislature in an election last month.
Two of the new lawmakers pledged allegiance to a "Hong Kong nation" and displayed a "Hong Kong is not China" banner as they took their oath.
In response to their action, the head of the city's Legislative Council Secretariat said he had no authority to administer the oaths to Yau Wai-ching, 25, and Baggio Leung, 30.
"As a member of the Legislative Council, I shall pay earnest efforts in keeping guard over the interests of the Hong Kong nation," Leung said, just before he took his oath.
He told Reuters on Tuesday that by the word "nation", he meant a body of people, not a country.
A third legislator, surveyor Edward Yiu, added a line about fighting for genuine universal suffrage at the end of the official statements. His oath was also not accepted.
The oath taking is an early test of the new legislators' determination to push independence issues in defiance of Beijing's steadfast opposition to any such suggestion.
Hong Kong's mini-constitution, or Basic Law, drawn up when the former British colony returned to China, does not explicitly forbid discussion of independence and it grants freedom of expression.
But its first article states that Hong Kong is an inalienable part of China.
Legislators must complete the oath, swearing allegiance to the Hong Kong Special Administration of the People's Republic of China, before they can take up their seats or vote.
Chinese foreign ministry spokesman Geng Shuang declined to comment on what the new legislators said but reiterated Hong Kong's status as part of China.
"Hong Kong is a special administrative region of the People's Republic of China. This is very clear and unquestionable, and will not change," he told a daily briefing.
Pronunciation
Later, scuffles erupted when the three activist politicians were locked out of a session that voted in a legislative president.
Leung shouted that staff were shutting him out illegally. Inside, guards shielded a senior pro-establishment legislator from angry democrats, some of whom tore up ballot papers.
Earlier, another source of controversy at the ceremony was that Yau and Leung pronounced "China" as "Chee-na," which some legislators said sounded like a derogatory term the Japanese used while they occupied China during World War Two.
The two said the supposed pronunciation was merely a quirk of their accents but a pro-establishment lawmaker, Priscilla Leung, said it was "very offensive" and "unacceptable".
Related reading
Hong Kong pro-democracy candidates win stunning victory
Yau also appeared to utter a profanity while reading "People's Republic of China" in English during the ceremony, but she later denied it.
The global financial hub was promised extensive freedom and a high degree of autonomy under a "one country, two systems" principle when the British handed it back in 1997.
But younger people have grown frustrated at stalled changes, sparking unprecedented calls for various forms of greater autonomy, from self-determination to independence.
Immediately after the election in September, Beijing's main office overseeing the city, the Hong Kong and Macao Affairs Office, said it "resolutely opposed" any form of independence.
On Wednesday, some other pro-democratic lawmakers shouted slogans and held up props, including a yellow umbrella, a symbol of 2014 pro-democracy demonstrations, but nevertheless had their oaths administered.
Senior officials and judges have warned privately that they fear Beijing may push legal changes on Hong Kong to effectively outlaw discussions of independence or self determination.
But debate of the issue in the legislature might be unavoidable, a pro-Beijing legislator said.
“We cannot disallow discussion of certain topics that are already widely discussed in society," said Starry Lee, leader of the largest pro-Beijing political party.
"We should not run away from relatively sensitive issues. But I do not believe radical actions will gain support from most people.”
