Michael Wood, chair of the judges, described A Brief History of Seven Killings as the "most exciting" book on the shortlist.
The 686-page epic, featuring more than 75 characters and voices, was "full of surprises" as well as being "very violent" and "full of swearing".
James was announced the £50,000 winner on Tuesday night at London's Guildhall. The 44-year-old, from Minneapolis, is the first Jamaican author to win the Man Booker Prize.
Receiving the award, he said a huge part of the novel had been inspired by reggae music.
Set across three decades, the novel uses the true story of the attempt on the life of reggae star Marley, referred to as “the singer”, to explore the turbulent world of Jamaican gangs and politics.
Wood said the judges had come to a unanimous decision in less than two hours.
He added: "It is a crime novel that moves beyond the world of crime and takes us deep into a recent history we know far too little about. It moves at a terrific pace and will come to be seen as a classic of our times."
Of the book, the New York Times said: ‘It’s like a Tarantino remake of “The Harder They Come”, but with a soundtrack by Bob Marley and a script by Oliver Stone and William Faulkner... epic in every sense of that word: sweeping, mythic, over-the-top, colossal and dizzyingly complex.'