Around 150 people were evacuated from the store in High Lane, and two people who were in the store have been taken to hospital.
Fifty people from surrounding buildings were also evacuated.
At least seven people were given paramedic help at the scene earlier.
West Midlands Fire Service have now said that "Longbridge will shortly reopen".
Officers from West Midlands Police were called to the area around 10am on Friday 20th May, and have described it as an "ongoing incident".
West Midlands Fire Service teams say their specialist officers are now using air monitoring equipment inside the store, to try to establish the cause of the incident. They said:
'We are working closely with the store management, as well as our ambulance and police colleagues and Birmingham City Council.
'A nearby college building was not evacuated, but advised to keep windows closed as a precaution'.
West Midlands Police said: "We have specialist officers working with colleagues across the emergency services to establish exactly what has happened.
"A cordon has been put in place and we'd advise people to stay away from the area.
"Anyone who has been in the area and is feeling unwell should seek medical advice as a precaution."
West Midlands Fire Service teams are treating this as a 'Hazardous Substance' incident.
They said: "We were alerted to an incident at the Sainsbury’s supermarket on High Street, Longbridge, Birmingham.
"A number of people had reported feeling unwell, with apparent breathing difficulties.
"Two fire engines and one of our 4x4 Brigade Response vehicles responded, from Northfield and Kings Norton fire stations, together with a number of officers and colleagues from the police and ambulance services."
They added: "The first crew was at the scene within three minutes of being mobilised.
"The store had been evacuated before we arrived.A 50-metre cordon was put in place around the store.
"Other premises within the cordon were also evacuated".
In a later update, West Midlands Fire Service said in a tweet: "Update: High Street, #Longbridge will shortly be reopened.
"The cordoned-off area is being reduced to include only Sainsbury's. The store will remain closed for the time being pending our ongoing investigations."
Operation Station Commander Kris Darnley from West Midlands Fire Service told ITV News Correspondent Charlotte Cross that fire teams had carried out a series of tests at the site to establish any cause.
One eyewitness, Laura West, told ITV News Central: "I work opposite the Sainsburys/retail park in The Factory.
"I arrived to work at 9:40 this morning and at around 9:55 three ambulances arrived.
"You could see a man lying on the ground by the supermarket but he seemed responsive.
She went on to say:
"At around 10am the first fire engine arrived and they began cordoning off the street so people couldn't enter the area.
"About 15 minutes after that, the other fire engine and an incident van arrived as well as a work van with freezer repair services advertised on it.
"Large areas of the area surrounding the incident are now taped off and there is a large police presence'.
In a statement, West Midlands Ambulance Service said: "A total of seven patients were assessed by the ambulance service after displaying minor respiratory problems.
"Two women received treatment on scene before being conveyed by ambulance, in a stable condition, to Queen Elizabeth Hospital Birmingham for further checks.
"The remaining five patients were later discharged on scene.
"A number of other people at the scene, who were described as ‘worried but well’, received advice from the team of ambulance staff.
"This was a well-managed tri-service response which, together with the help from retail staff, saw the safe and speedy evacuation of a large number of people. Investigations into the cause of the incident continue...
"The public are advised to avoid the area whilst emergency services deal with the incident."
A Sainsbury’s spokesperson said: "Emergency services attended our Longbridge store this morning.
"Safety is our highest priority and the store is closed while we assist the police with their investigations."