Offenders would see a rise from three to four penalty points on their licence and a rise in fines from £100 to £150.
The plans, to be consulted on in 2016, would also see HGV drivers hit with the same increased fines and penalty points doubled, from three to six.
An educational course to try to change behaviour could still be offered to first time offenders.
Road Safety Minister Andrew Jones says HGV drivers are being specifically targeted to reflect the potential consequences of an accident.
“We are increasing the penalty points for HGV drivers because these are big, big vehicles up to 44 tonnes in weight,” he said.
“The consequences of people not paying attention behind the wheel of these vehicles can be significantly greater, so that’s why we are treating them differently.”
Officers from the Central Motorway Policing Group, based in Birmingham, are catching offenders on a daily basis and welcome the proposals.
Superintendent Paul Keasey says the rise in social media use is one reason why drivers are still using their mobile phones on the road, over 12 years since it became illegal.
“We almost want to know everything instantaneously and that hasn’t really transferred onto how people should be driving,” he said.
“One of the big things is they don’t realise what a distraction it is. If you actually take your eyes off road for a second your braking distance is significantly reduced.
“We are absolutely fully behind the government and our partners in preventing this type of occurrence happening.”
Transport Secretary Patrick McLoughlin says the proposals are part of a wider package of measures to improve road safety.
“We have some of the safest roads in Europe but we are always looking for ways to improve that record,” he said.
“Using a mobile phone at the wheel is reckless and costs lives – I want to see it become a social taboo like not wearing a seatbelt.
“We will take action to tackle this persistent problem, with an emphasis on the most serious offenders.”
In 2013 the penalty for mobile phone use while driving was increased to £60.
In 2014 the use of a mobile phone was a factor in 21 fatal accidents and 84 serious accidents.