The gunman then allegedly walked towards the tent and opened fire, wounding the 3-year-old boy's parents.
Witnesses said the shooting sounded like "firecrackers".
Some 15 people were wounded during the mass shooting at the three-day Gilroy Garlic Festival in northern California, some of them hurt in the crush of bystanders trying to flee, police said. He appeared to have randomly targeted his victims with a rifle.
Witnesses reported confusion and panic as shots rang out at the festival in the city of 50,000 located about 80 miles (176 kilometers) southeast of San Francisco.
Smithee said that search warrants had been obtained for a home in Gilroy associated with the suspect and a vehicle that police believe he drove to the festival.
The shooting occurred during the three-day celebration featuring food, cooking competitions and music that attracts more than 100,000 people.
The shooting rampage began at 5:41 p.m. on the north side of the festival, Chief of Police Scot Smithee said at a press conference late Sunday.
"We are again continuing the investigation, and we plan on being out here all night", said the mayor.
Police declined to name the fatalities on Monday, but one of them was identified by his grandmother in an interview with the local ABC affiliate as Stephen Romero, 6.
"There's nothing I really can do besides try to be with him until I can put him in his resting spot, wherever that is", his father, Alberto Romero, told NBC Bay Area. "That's all I can say". One person was in critical condition, a hospital spokeswoman said.
President Donald Trump described the gunman as a "wicked murderer" during an event at the White House, and asked Americans to pray for the victims. "We will continue to support our neighbors in any way we can". At least five have been released.
"Tonight, (California) stands with the Gilroy community. My office is closely monitoring the situation", Kamala Harris tweeted.
Smithee said police responded within one minute and killed the suspect.
"Grateful to first responders who are on the scene in Gilroy and keeping those injured by such senseless violence in my thoughts".
The band TinMan was starting an encore as the shots ran out.
Marquez said her niece told her she made eye contact with the gunman, but that he didn't fire.
Despite his own wounds, Bates turned back and ran in the direction of the shooter to save his friend.
Van Breen, from nearby Santa Clara, said he heard someone shout: "Why are you doing this?"
Van Breen and other members of the band dove under the stage. The audience also started to scream and run.
Law enforcement have already recovered a firearm and rifle ammunition from the shooting, a law enforcement official told CNN, but they have not provided the exact make of the firearm used.
13 year-old Evenny Reyes told the paper that "we were just leaving and we saw a guy with a bandana wrapped around his leg because he got shot".
"He was joyful, always wanted to play, always positive", he said, adding that the boy's mother has been placed in a medically induced coma after being shot in the stomach and her mother was being treated for a gunshot wound in the leg.
As police swooped on the gunman and shots were fired, festival attendees dropped to the ground, Mr Paz said. He cut through a fence to enter.
Taylor Jackson was working at a booth drawing caricatures when she heard gunfire, saw people running and "ran for the hills". The killer was later revealed to be a 19-year-old, identified on his social media account as being of Italian and Iranian descent.
- Defoe strengthens Euros case
- Oppo shows off new edgeless "waterfall screen"
- Klitschko faces must-win bout against Fury
- Defeated Thomas still believes he can win another Tour
- What is Apple Card and Daily Cash?
- WhatsApp buiding desktop version that can be used without phone
- Apple is acquiring majority of Intel's smartphone modem business
- Talent Not an Issue for England's New Manager
- Fact check: No, African Americans are not happy with Trump
- Google confirms Face Unlock and Soli gestures for the Pixel 4