"After we knew our algorithm was pretty close to what we wanted,
we converted the process to a giant Hadoop Map/Reduce program
running on a cluster of Amazon EC2 servers for about 20 days to
get the final results for the first version. Smaller
optimization processes still run continuously to test new ideas
and refine the model"
"Unlike static rules or blacklist-based methods of detecting spam
, all of the major Omnivore systems are learning algorithms that
keep up with changing user behavior without losing their
predictive power"
- MailChimp's Project Omnivore: Declassified | MailChimp Email Marketing Blog
http://r4.sharedcopy.com/47hev