The U.S. radio market is a competitive business that is undergoing significant change. Operators are being pressured by new rivals in digital, satellite and Internet radio, including subscription services, such as Pandora and Spotify. More than 15,000 stations vie for $18 billion in annual U.S. radio advertising spend – a figure projected to grow by less than 1 percent annually through 2021.
To generate revenue, radio operators need to build listener numbers. To attract listeners, they need to play the specific artists and songs their target audiences want to hear. Years ago, radio playlists were created largely on gut feelings, based on personal experiences or industry connections. Today, programmers curate their playlists carefully, tapping data from relevant reports and research.
The challenges stations face are determining which data sources to trust – and then how to put data into action. Different services have different business models, and they generate different insights about what may be popular in certain markets, at certain points in time. Stations themselves collect their own data, but it is often stored in analog fashion, on spreadsheets and in file folders, making it hard to analyze using cloud technologies at scale.
Streamlining a Media Company’s Use of Data
One of America’s largest radio providers embarked on a project to create an analytics function that would be the envy of the industry. The company engaged CTP to design and build an enterprise-grade analytics solution to support internal program directors, as well as key business partners: large U.S. record labels, artist agents and brand managers.
The multi-phase project incorporated initial steps to analyze existing methods of data collection and analysis, create buy-in with C-suite stakeholders on a plan and design workflows facilitating new methods of analysis. The final step would be to deliver a solution using cloud native applications to help the organization’s clients modernize their processes.
Laying the Groundwork
CTP combined agency level experience design with cloud adoption best practices to design and develop a solution that delivers great customer experience, and a cloud deployment that optimizes performance and cost.
To get started, CTP led an Experience Design (XD) Discovery Workshop with key stakeholders — including the CEO and executives with industry experience — to unlock key goals and problems the media firm was facing. CTP identified the industry personas – “power players,” “industry veterans,” “industry front-runners” and “program directors” – who each had different motivations and different experiences leveraging analytics and technology. We led interviews and user testing with the key personas and shadowed them to better understand their day and life. The common thread was that personas tended to use data inconsistently, usually as a secondary tool to validate assumptions. What they needed was a way to easily visualize data regarding a song’s performance against other key indicators and overlapping audience profiles.
CTP moved on to a prototyping phase. The team collected rows of data, using Google Sheets and JSON, to scope out a model to visualize song trajectory. They explored several visualizations to display the data, and created a UI, leveraging geomaps, bar graphs and trend lines. This led to an evolved workflow used to benchmark artist and song lifecycles.
CTP discovered that the biggest challenge was a lack of “clean data,” since it all was not created equally. For example, market boundaries and time windows for sales were not defined consistently across retail, broadcast and digital channels. To extrapolate cross-platform insights, CTP looked at ways to re-architect existing data strategies. Engineers helped the client scale a data “pool” as a precursor to building standard metrics their teams could rely on.
In parallel, using the existing Excel worksheets to develop a re-imagined application user experience, CTP created a secondary mechanism validating the data with industry veterans. This would help train their data models, reduce complexity and build intelligence over time. As a result, the team scaled back and simplified the UI to provide the right data at the right time to the right person – in the appropriate format and medium or channel.
Delivering a Solution
CTP then delivered a minimum viable product (MVP) for the music analytics dashboard application. This solution consisted of four components.
- Music Analytics Dashboard – providing views into artist metrics and song lifecycles
- Music Research Dashboard – combining historical data from multiple sources, ongoing research and predictive metrics enabling local stations to collaborate, develop music schedules and make better data-driven decisions in a single unified view
- Serverless ETL pipeline – meshing data from disparate sources that, in turn, powers the application
- Cloud Search – enabling an easy form of navigation and discovery across the MusicLab user interface
The Music Analytics Dashboard application follows a traditional three-tiered architecture with a presentation layer, a business layer and a data layer. The presentation layer is an AWS Elastic Beanstalk application, running node.js and built using React framework components. The business layer is more in line with the current trends of serverless architecture – written using Amazon API Gateway and AWS Lambda, along with custom authorizers, and integration with Azure Active Directory via their OAuth interface. The data layer is a multi-availability zone AWS RDS instance using the PostgreSQL engine due to the relational nature of the data.
The serverless ETL pipeline is a MapReduce pipeline built using serverless components instead of Hadoop. Key features include: abstracted data ingestion that connects to FTP, S3, Google Cloud Storage, SQL databases, SOAP services, Windows network shares and HTTP data sources; Node.js modules for all things ETL; and async streaming between different data protocols.
The Amazon CloudSearch component provides the application with an autocomplete search bar, as well as a full search results page that includes fuzzy matching.
Staying Ahead of Trends
Determining which songs will be hits remains an inexact science. A “sound” that connects with a particular audience one year may fall flat in coming years, forcing radio programmers to pivot with new strategies. While these situations will continue to play out over time, radio executives are working hard to analyze song cycle data so they can stay ahead of breaking trends. Using the cloud, and leveraging better data ensures that stations are playing the songs their listeners want to hear.
