iStock_000000051702XSmall
After we knew our algorithm was pretty close to what we wanted, we converted the process to a giant Hadoop Map/Reduce program running on a cluster of Amazon EC2 servers for about 20 days to get the final results for the first version.  Smaller optimization processes still run continuously to test new ideas and refine the model.
Unlike static rules or blacklist-based methods of detecting spam, all of the major Omnivore systems are learning algorithms that keep up with changing user behavior without losing their predictive power.
- Web annotation on MailChimp's Project Omnivore: Declassified | MailChimp Email Marketing Blog

Share this annotation

Post to Basecamp Project Update Twitter Bookmark on Del.icio.us Send E-mail Post to a Blog Post to Backpack Post to Trac Post to Bugzilla Post to a Tumblr Update Friendfeed Posterous

paste in your blog
 
paste anywhere: IM, mail
 
give this link to a friend
 

Tags: aug_10, mailchimp.com, annotations
Comments are allowed
This copy is published

Note: "This copy is kept-secret" would mean its URL is not published, but anyone knowing its URL can still view it.