In an effort to beat benchmarks, investment companies are looking at the entire dataset of Twitter, known in the business as the “full firehose”.

Few people can manage the sheer scale and storage challenges that come with it, not to mention the costs. You could start searching the social media stream using a hashtag approach.

Peter Hafez, chief data scientist at data analytics firm RavenPack, knows how tricky it is to process large volumes of noisy unstructured data.

He remembers a small hedge fund, which tried the hashtag approach on “gold”, hoping to create a gold sentiment indicator to trade the related futures contracts.

Unfortunately, their algorithms didn’t take into account how often gold is mentioned during larger sports events like the Olympic Games; and they ended up not being particularly successful.

Read more: Big data, machine learning and AI used by hedge funds to deal with Twitter ‘firehose’

Don’t forget to share this via , Google+, Pinterest, LinkedIn, Buffer, , Tumblr, Reddit, StumbleUpon and Delicious.

Published by Mike Rawson

Mike Rawson has recently re-awoken a long-standing interest in robots and our automated future. He lives in London with a single android - a temperamental vacuum cleaner - but is looking forward to getting more cyborgs soon.

Leave a comment

Your email address will not be published. Required fields are marked *

Hedge funds use big data, machine learning and AI to deal with Twitte…

by Mike Rawson time to read: 1 min
Hi there - can I help you with anything?
[Subscribe here]
More in Machine Learning, News
Driverless car deliveries
Japan To Roll Out Driverless Cars To Deliver Goods

Japan will be the first country to use self-driving cars to deliver items purchased online. The service, called “on-demand service...