The figures emerged after a search of some 8,000 removal records dating back to 2002.
The investigation was part of efforts to tackle the scandal of people arriving legally before 1973 who more recently have been told to leave or been denied jobs, healthcare and homes because they do not have documents proving their right to live in Britain.
Mr Javid told the Commons Home Affairs Committee that officials had so far found 63 people who may have arrived from the Caribbean before 1973 – the cut-off date for residence rights – who had been made to leave.
The figures were not finalised, but 32 so far were foreign offenders that would have gone through legal processes before being deported. He added: “If they feel they have been wrongly deported, it is up to them to do something about it.
Sajid Javid said 63 deported people may have arrived in Britain before 1973
But the Government would “proactively” seek to contact the other 31.
All but one left the UK “voluntarily” after being told to leave, he said.
The Home Office will work with the people’s governments to track them down and ask if they wanted their cases re-examined. He added the figures for potential wrongful deportations were not final.
Figures released in documents provided to the committee showed that a dedicated helpline set up after the Windrush furore erupted has received more than 11,500 calls. Of those, nearly 5,000 were identified as possible Windrush cases and referred to specialist caseworkers.
Mr Javid denied there was a “systemic” problem in the Home Office, but accepted that in Windrush cases people had faced “too large a burden” in proving long-term residency.