The film starts off on Hanukkah in 1885, opening in the village of Shostka, Russia, and it shows the story of the life of a family of Jewish-Russian mice who emigrate after their village is destroyed by Tsar Alexander III's Anti-Jewish pogroms. Elsewhere, in Sicily, mice were being terrorized by a mafia tabby cat by the name of Don Tabilioni. Believing in the American dream, they head to New York City because (as a song repeated early in the picture goes) "there are no cats in America, and the streets are paved with cheese." Once there, they immediately discover that there are indeed cats in America (and plenty of them), and begin living in a typical late 19th century immigran manner – working in a sweatshop, living in horrible conditions, and submitting to a feline protection racket as an alternative to being eaten by the cats.
The film follows Fievel Mousekewitz, who, along with Heckle and Jeckle, is separated from the Mousekewitz family after falling overboard the SS Austria as it approaches America after departing the Port of Hamburg. Stuck inside a floating vodka bottle, Fievel, Heckle and Jeckle are carried to Liberty Island, where they are met by a pigeon named Henri, who is with a French delegation who are erecting the Statue of Liberty. Heckle is at first doubtful that they can even find Fievel's family in America, but Henri encourages them to try. Fievel, Heckle and Jeckle make there way to New York City, hoping to consult with Immigration, but are waylaid by Warren T. Rat, a Shakespearean reciting conman rat who sells them to a sweatshop. Fievel, Heckle and Jeckle escape, befriending an Italian mouse named Tony Toponi (who nicknames him "Philly") in the process. Tony and Bridget, an Irish mouse with whom Tony is friendly, begin a search for his parents, who believe that Fievel, Heckle and Jeckle drowned at sea.
During this time, the mice of New York decide that they are fed up with the continuous attacks by cats (and paying Warren for useless protection), and must find a way to defeat them. A wealthy mouse socialite named Gussie Mausheimer holds a rally to bring more mice to their cause. However, no one is sure just what to do to get rid of the cats. It is Fievel who suggests a plan to build a giant "Mouse of Minsk" (based on Russian folklore).
They make their way to an abandoned building along the Chelsea Piers that night and inside begin to work building their creation. Fievel, Heckle, Jeckle and Tony however are late. Running hard to get to the docks, Fievel and the others are sidetracked by the strains of a violin being played from a sewer grate. Believing it's Papa, they make there way into the sewer, only to find out that the music is coming from Warren and a gang of cats. Warren is actually a cat in a rat disguise and is running a protection racket as the gang's boss. When they discover Fievel, Heckle and Jeckle, they are captured and imprisoned in a cage.
Fievel, Heckle and Jeckle end up befriending one of the cats, Tiger, who admits that he's a vegetarian. As the 4 talk, they discover that even though they are a cat and mouse, they share some of the same likes and dislikes. After talking, Tiger lets Fievel and the others go. However, there escape triggers an alarm, and Fievel, Heckle and Jeckle are chased to the New York Harbor where Fievel reveals Warren's identity as a cat with a little help from Tony. When the mice in their encampment refuse to give in to Warren's demands, he lights a match and attempts to burn them alive. However, Fievel intercepts the match, and uses it to help jumpstart their 'secret weapon' – a gigantic creation based on the story of "The Giant Mouse of Minsk," armed with fireworks provided by Chinese mice. The giant machine chases the cats down the docks and forces them onto a tramp steamer bound for Hong Kong.
It appears the mice have won out, but a remnant of the fire Warren started is still lit, and a spilled bottle of kerosene soon starts it to cause a blaze at the encampment. As the mice scramble for cover, firefighters are dispatched to the blaze to put it out. In the ensuing chaos, Fievel's family encounter Heckle, Jeckle, Tony, and Bridget, who are looking for "Philly". Tanya believes they're looking for Fievel, and Tony confirms that Philly's original name is indeed Fievel. Papa still believes that Fievel is dead, but Bridget gives Fievel's hat to Mama and she shows her husband (which he gave Fievel at the start of the movie), confirming that Fievel is alive. Tiger is also present, and overhears the conversation among the mice.
Shortly afterwards, Fievel wakes up on the streets with some orphaned mice, who tease him about looking for his family. Fievel believes them and sadly resigns himself to life as an orphan. The next morning, Fievel's family and new friends are searching for him together, everyone riding on Tiger, and Papa playing his violin. Fievel hears the music, and the Mousekowitz family is soon joyfully reunited. Bridget tells Tony that it is very beautiful to see a family together. Tony responds by saying, "Ain't nobody thanking me." Bridget thanks him with a kiss, while Tiger is very happy to have friends, "Little tiny friends." The movie ends with Heckle, Jeckle, Fievel, Tanya, Papa and Mama all riding on pigeons (much to Mama's dismay) to see the now-completed Statue of Liberty, which winks at them.
- The Story continues in Heckle and Jeckle's Adventures of An American Tail: Fievel Goes West.