Swimmers should be banned from crossing the Channel from Dover to Calais, according to the French coastguard.
People attempting the 21-mile challenge have to cross the world's busiest shipping lane.
Deputy director of the French Coastguard Jean-Christophe Burvingt told BBC South East's Inside Out he feared there could be an accident.
But Mike Oram, of the Channel Swimming and Piloting Federation, said the crossings were safe and well organised.
More than 500 vessels pass through the shipping lanes each day.
Mr Burvingt said: "This continuous increase of swimming in the Channel creates a danger which is getting more and more important every year.
"I think that there will be a collision. There will be an accident due to that [cross-Channel swimmers]."
France banned swimmers using the Calais to Dover route 17 years ago.
Kaimes Beasley, of the UK's Maritime and Coastguard Agency (MCA), said cross-Channel swimming was "as dangerous as trying to cross the M25".
Some ferry operators have also warned that a swimmer could cause a major accident.
Chris Newey, of DFDS Seaways, said: "We'd like to see some control over the numbers of people that are doing this activity, and when they're doing it, and some regulation around that activity from a health and safety perspective."
The Channel Swimming and Piloting Federation, which accompanies swimmers and is one of two regulatory bodies authorised to organise and assist swims and other crossings, said the challenges were as safe as could be.
"We are over the top with all our safety precautions," Mr Oram said.
"We have tracking systems on the vessels, we have everything else that is required, we report into the coastguards.
"They know well in advance exactly what we're doing and how we're doing it and there is a restriction on the number of pilot boats."
He added that the number of swimmers had reached a peak and figures had stayed roughly the same for the past four or five years.
No swimmer or support boat has ever caused an accident in the English Channel to date, but the MCA has said "swimmers should be persuaded to look for a challenge elsewhere".