Thursday, November 17, 2005

And the funkiest year ever was...

Warning. This post has a terribly high geek quotient. If plotting the frequencies of data about hip hop offends you in any way, please do not read on. OK. I warned you. Now I will tell you what the funkiest year ever was. (If you want to guess first, it might be more fun. Post your guess at the Riddim Method)

I recently returned to the data I culled from the The Sample FAQ to answer a question I have long pondered: "What was the funkiest year ever?" Because the Sample FAQ lists the year of the original songs that have been sampled by hip hop producers, we can construct a simple frequency count of the number of samples that used material from a given year. All I did was add up the number of times that people sampled songs in each year and plotted the frequencies in Excel so that the categories are years and the values for each category are the number of times songs from that year were sampled. If none of this makes any sense to you, just think of this as "freq.-ing the funk".

The resulting distribution will not surprise people familiar with hip hop, sampling or the explosion of amazingly funky music in the early 1970's. As you can see, the most heavily sampled period (represented by the peak in the distribution) occurred between 1970 and 1975. But the actual peak occurs with 872 different songs that sampled material released in 1973. So that's it. According to this source at least, 1973 was THE FUNKIEST YEAR EVER! (If you accept a few conditions and caveats - see below).

As I looked at this distribution, I noticed a few things. While the historgram has the shape of a normal distribution, it is probably not when viewed properly (See comment below). While normal distributions are characteristic of random occurrences, hip hop producers' choices of samples are anything but. It is not as if people randomly selected old songs to sample. Just the opposite; they sampled a very small subset of the very funkiest old songs (particularly those with open drum breaks). As a result, this distribution is perhaps better thought of as a series of votes. In this election of sorts, the underlying dimension being voted on is something abstract like "songs that have funky breaks" or "funky ass songs".

Thinking of the distribution this way makes sense to me. Its like each time someone wants to sample something, they go wading into the records looking for a break, stab, slice or some other crazy sound to use in their own music. Each time they sample something, that thing gets a vote. Over time, some things tend to get more votes than others.

Note that if we think of it this way, it seems likely that the distribution has a long tail. James Brown gets the lion's share of votes (along with a few other bands like Parliament), but most get only one or two. (Phew, glad to see that long tail back).

But that is not quite right either, because what determines what people are looking for to begin with? Other people. If it were just funky breaks that people were looking for, the curve above would be a line going straight up to the right. There are plenty of breaks in hip hop. If you are looking for the funkiest stuff, why not just sample the samples, and so on? That would give you an increasing pool of stuff to sample and thus the line going up and right. Well, there are rules against that you see. Cultural rules. This might be of interest to my ethno pals who are sometimes asked to defend methods that start by asking people what rules they use.

Methodological musings aside, as I looked more closely at the distribution, I noticed something really strange: The peak in 1986. "Why," I asked, "were there so many samples of songs released in 1986 when clearly the 'best' sample source material was released in the early 1970's?" And then it dawned on me, "The funky drummer" break.

Although the most heavily sampled drum break of all time was actually released in 1970 on a King 45 (6290), it was re-issued (along with a lot of other classic JB material on the now infamous In the Jungle Groove LP. Guess when. Yup. 1986.
Of course, it was a heavily sampled and played record.
Now this is pretty interesting on its own (the fact that a single re-release can make such an impact on a cultural practice). I also wondered if it might reflect some error in the data. Since the sample FAQ includes tracks made by some of the biggest collectors and beat junkies in the world, I figured a bunch of them would have sampled the original 45 and went to check that this was reflected in the data. When I looked back, I realized the sample FAQ listed every instance of the Funky Drummer (and a few other tracks from the comp) as 1986. Acording to this, not one person sampled the original 45. It seems unlikely that in a list that includes all best producers in the history of hip hop not one would have sampled the original. Being a good riddim methodist (I thought B.C. was a Jesuit school), I decided to correct the data.

Making the conservative guess that half these producers would have used the original, I went back and adjusted the frequencies by allocating half of each track that was on the 1986 "In the Jungle Groove" comp to its original year. That gives us a very slightly more normal distribution.

In addition to being a hell of a lot of fun, I hope this little post points out the value of having ethnographic knowledge about your data. If all we know is the distribution of something in a population, we can do lots of cool things (like finding out what else it is correlated with). Without some ethnographic knowledge too, its harder to make claims about why.

This also points out that the data at the sample FAQ contains some potential gaps and omissions (raising concerns about others who may want to use it for academic research). On that point, let me offer another caveat. This distribution was based on the data on Funk and Soul at the Sample FAQ and might look somewhat different if it included the Jazz and Rock data. Feel free to use this data and my graphs freely in your work. If you do, just cite me. Something like this would be nice.

Foster, P. (2005) Hip hop samples by source year. Data published at

Or cite the Sample Faq. They are the ones to thank anyway.

Here's the data in Excel format


Anonymous cottoncandyhammer said...

head 'sploded!

4:54 AM  
Anonymous Anonymous said...

Got here via boing boing and found this very interesting although I didn't read it all!
Initially I thought this would be about what year you thought the funkiest hip hop came out, which for me peaks around '88/89 with stuff from De la soul, NWA, Tribe called quest etc.'s about the funkiest year for samples and I think your chart feels absolutely right without looking at all the data behind it.
Now if someone could do this for house music, I bet it would peak around '80.

6:57 AM  
Anonymous Len said...

You could also look at the data as the most sampled artist. My guess is that would be James Brown.

7:58 AM  
Blogger Rog said...

I really dig the post, but I think your analysis of the distributions is a little off base. I wouldn't say that these data are normally distributed. You basically have a histogram here, to figure out if you have a power law ("long tail" as you call it, but please note that many distributions have a "long tail") distribution you should sort the years according to frequency. 1973 will be first with ~870, 1972 is next etc.
Then, plot that on a double log plot and see how straight that line is.
Do you have the data posted anywhere?

12:40 PM  
Blogger Pace said...

Thanks for this clarification! I'll try to redo the analysis soon. In the meantime, the data is posted at the end of the entry in excel format.

10:28 AM  
Anonymous Anonymous said...

Give your Home a new look with window shutters. Your home can benifit from window shutters by adding curb appeal to your house.

2:43 PM  
Anonymous Anonymous said...

Handy man and Repairmen

Can You Fix this and Can you fix that well

estimated construction cost of a residential home foundation Advice at . If you are Looking for estimated construction cost of a residential home foundation Then Here it is

12:09 AM  
Anonymous Anonymous said...

shutters Tips for building or remodeling your dream home.

10:36 AM  
Anonymous Anonymous said...

News News News

Wow What is going on with all these homes with severe mold problems. I cant believe the problems we are having in our homes today. It has always been here. ask about Household Mold

12:30 AM  

Post a Comment

<< Home