HMRC has made over 40,000 requests to view people's private communication details in three years, a Freedom of Information request has revealed.
The 41,351 requests includes information covering text messages, emails and phone calls.
Only two police forces, the Met and Merseyside, made more applications for private data, according to civil liberties campaign group, Big Brother Watch, which obtained the figures for 2009 to 2011.
Emma Carr, deputy director of civil liberties campaign group Big Brother Watch, said: 'These figures show the taxman uses snooping powers more often than nearly every police force in the country. So it's strange that nobody has mentioned the government's Communications Data Bill will give the taxman access to who you email or chat online with and what websites you visit.'
Home Secretary Theresa May is spearheading the introduction of the controversial Communications Data Bill which will force all service providers to retain information for a year.
An HMRC spokesman said: 'HMRC uses Regulation of Investigatory Powers Act (RIPA) powers in support of its investigations into criminal attacks on the UK tax system.'
He said it only used RIPA powers 'where we have a criminal investigation underway which is where we suspect a fraud has been committed', adding that of the 14,381 interventions in 2011, all of them related to '5,005 individual communications information requests'.
The government recently published its Levelling the Tax Playing Field document, which set out the progress made in its drive against tax avoidance.
Among its successes are six corporate tax loopholes closed, a move which it says has 'protected over £1bn in revenue and yielded over £500m.'
Since 2010, HMRC has won more than 50 tax avoidance cases.