Fredette was born February 25, 1989, in Glens Falls, New York, to Al and Kay Fredette. Named James Taft, Fredette later received the nickname "Jimmer" from his mother. Jimmer describes his older brother TJ as his biggest fan and supporter. TJ, whose song "Amazing" was written for his brother, said, "I see him play, and it gives me chills sometimes when he hits some of those big shots and the crowd is going crazy." An unusual set of Fredette fans reside in the Mount McGregor Correctional Facility, a medium security prison, where he and his older brother played games with the inmates. Fredette credits these experiences as helping with his focus in rough situations. Jimmer is a senior at BYU with dreams of being an NBA star.