The Information Commissioner’s Office said the loss, which took place last August, broke data protection regulation, and ordered the Home Office to sign a formal undertaking to improve procedures.
The memory stick was left in an unlocked drawer by an employee of PA Consulting. The firm subsequently lost its £1.5 million contract with the Home Office, but retains a role on the £4.7 billion ID cards scheme.
Among the sensitive data on the stick were the names, addresses and release dates of all the prisoners, including 10,000 serious offenders.
Mick Gorrill, assistant information commissioner, said: "This case was serious because it involved thousands of individual records, which contained sensitive information on people serving custodial sentences and others previously convicted of criminal offences.”
"Even though a contractor lost the data, it is [the Home Office] which is responsible for the security of the information."
A Home Office spokesman said the department is “committed” to data security. "We have made good progress to improve data security and we will continue to work closely with the Information Commissioner's Office to ensure that our systems are as robust as possible."
Separately, it was announced last week that Christopher Graham, director general of the Advertising Standards Authority, has been nominated as the preferred candidate for information commissioner when Richard Thomas steps down in June.