Sentiment analysis, machine learning open up world of possibilities
When a person feels sufficiently wronged to lodge a complaint with the Consumer Financial Protection Bureau (CFPB), there’s likely to be some negative sentiment involved. But is there a connection between the language they use and the likelihood they will be compensated by the offending company?
At the upcoming Sentiment Analysis Symposium, I will discuss how machine learning and rule-based sentiment analysis can support each other in a complementary analysis, and produce actionable information from large amounts of free form text. In this case, machine learning and sentiment analysis could improve and evolve the CFPB’s ability to assess consumer complaints.
This is accomplished by identifying patterns between degrees of negative sentiment expressed in free-form consumer complaints. A model which generates rules based on this free-form text, where the related companies ended up paying out compensation as a result of the complaint. These machine-generated rules indicate patterns in the free-form text which tend to only be present in the cases of monetary compensation.
Examples include types of lending and retail companies associated with the lending but not present in the structured data. For example, if someone lodges a complaint about bank fees, and uses a derivative of the term “steal”, it is more likely to be associated with some kind of financial recompense. This goes beyond traditional sentiment analysis, identifying key negative terms, in a particular context, to highlight patterns associated with a result.
Visual analytics provides these newfound insights with illustrative structure – a previously hidden, yet incredibly valuable, map of areas of concern, including predatory lenders or credit card companies with substandard customer service.
Being able to rapidly identify and visualize key information – to anticipate something like consumer sentiment – has huge implications for the entire global economy. But the speed at which the analysis can be set up and operationalized against the data also makes it a game changer for predicting, preparing for and responding to population and infrastructure threats, such as natural disasters and public health crises.
I asked Sentiment Analysis Symposium organizer Seth Grimes, a thought leader in the text analytics sphere, for comment. Reflecting a perspective that aligns with my own, Seth says, "It's cool that SAS is able to show that detecting signals in consumer, health and behavioral and correlative big data can help agencies (and corporations) meet mission and public needs while saving enormously on information processing costs. I'veworked with SAS for decades and recognize SAS as a leader in text analytics and sentiment analysis, so it's great to see it applied, per Tom's talk, for public benefit."
With a natural disaster, officials could use machine learning and sentiment analysis to visualize patterns between mood-state indicators (social media posts geo-tagged near the affected area) and existing field data (why and when certain people visited a particular clinic on a given day) to better understand how to allocate resources and sharpen future preparedness efforts. For example, the analysis may indicate that there were a number of cases where individuals required oxygen, or access to certain prescription drugs such as warfarin. Knowing this, and providing access to these resources during a crisis will help to preserve lives.
Similar analysis could similarly improve the ability of epidemiologists to catch and fight infectious disease outbreaks early on, and of public health researchers to identify prescription drug users at-risk of overdose.