By Vikas Shanbhogue, Head of Platform
PlaceIQ has built a technology platform that contextualizes a vast amount of location data, in order to draw conclusions about consumer behavior based on physical locations. This process empowers data analysts and marketers to apply location and behavioral-based insights to strategic marketing efforts across industries like auto, retail, CPG and beyond.
While the value of actionable location intelligence is widely understood and accepted, the technology behind it is less talked about and often remains a point of curiosity.
This blog will dive into the nuts-and-bolts of our location intelligence technology, and shed light on the breadth of problems our team of engineers and data scientists are working to solve. Warning: this blog will get technical, so look for indicators of some complex sections.
PlaceIQ engineers built a domain-specific language called PIQL to efficiently understand consumer behavior through the lens of space and time. PIQL was specifically created to translate location data into relevant means for marketing and analytics. Using this language, we can scalably segment and score location data to break down and understand human behavior. As a result, our team of analysts — whose educations vary from anthropology to cartography — is able to evaluate, build and explain audience segments without highly technical engineering backgrounds.
Interesting and common use cases addressed by PIQL:
PIQL can easily identify audiences that:
— Work no more than a 3 minute walk from a Starbucks in Boston.
— Used to visit a major electronics store regularly up until a month ago but then switched to a top competitor.
— Go to a fast food restaurant before going to the movie theater.
— Go to a fast food restaurant once every week over the course of a year.
— Exhibit no brand loyalty and are indifferent between big box retailers.
PIQL can also categorize locations that:
— Represent a luxury clothing brand (from a predefined list) under the Retail->Clothing->Luxury segment.
— Are parking lots and are from high fidelity data sources (such as our internally created basemap) to a segment called PlaceIQ>Parking_Lot.
The above segmentation examples can then be used downstream across several applications, including media targeting, foot traffic analysis, out of home attribution and retail site selection.
Building audience segments is a core process at PlaceIQ, so automating the technical details allows our team of engineers to focus on consistently redefining the actual attributes that make up segments, and improving technical features.
Numerous iterations occurred before the decision was made to build our own domain-specific language. When we first started, we followed an “ad-hoc scripting” approach where we modified scripts in languages such as Python, Java, Hive and Pig whenever we needed to create a new segment. Unfortunately, this approach was error-prone, added complexity to the code base, and slowed down turnaround times.
Instead, we opted for an expressive language that reads a list of location rules, and assigns consumers to pre-defined segments based on their movement activities. In addition to standard audience segments, PIQL offers the flexibility to create custom, marketer-defined segments for specific use cases.
Our first stab at this involved a very basic grammar along with a limited set of operations that we call “legacy” PIQL. While this was sufficient for a limited set of use cases, we found ourselves having to run custom flows for more nuanced and complex audiences. To remedy this , we revamped the language to allow for:
1. More expressive rules (complicated time-based and logical constraints)
2. Supporting user defined operations for custom logic (examples: current distance from a certain building, or was observed in two different cities within 10 days of each other)
3. More user friendly syntax (simpler time specification, wildcards when specifying names in a taxonomy)
Audience segmentation and custom rule creation do not require technical engineering consultation since PIQL’s high-level language and syntax can be easily understood by non-engineers. Consequently, audience generation and deployment times are significantly faster.
PIQL is used to automatically assign consumers to high-level segment categories in a way that is:
1. Easy to write – Rules are intuitive (non-technical) and have clear semantics
2. Repeatable – Can handle new data sources
3. Reliable – Adding new rules shouldn’t break existing flows!
Broadly speaking, PIQL creates segments for two kinds of data:
Physical Location Data: These segments refer to real-world locations like retail stores, concert halls, or a consumer’s workplace. Our team of cartographers has created hand-drawn outlines (called Polygons) of these locations to determine the type of location and its specific boundaries. For example, if a location is a big box store, PIQL will include it in a Big_Box_Retailer segment. If it has been classified as a home or residential area, it will be filed under the Work_Dwell segment.
Consumer Behavior Data: These segments are created based on consumer behaviors and attributes. For example, if a group of people have visited a fast food restaurant 30 times or more in the past year, they will be included in the Frequent_Fast_Food_Eater segment. To go a step further, PIQL might identify a subset of consumers who also regularly attend concerts, and will assign them to a Concert_Viewer_And_Fast_Food segment.
These two data types work together to create hyper-defined audiences that marketers can use for media, insights, analytics, and to make complex enterprise business decisions such as figuring out where to open a store based on foot traffic or understanding the characteristics of people that have been exposed to a billboard.
The Nuts and Bolts
Let’s walk through a real-world example of how PIQL works. We’ll start with an example segment of coffee shop visitors in Boston. The data could be pulled from any location-based dataset. For this example, we’ll work with a fictional dataset from Boston, referred to by PIQL as Boston_OpenData_Retail.
Our first goal would be to aggregate the Boston coffee shop locations, and give them an appropriate tag (i.e. PlaceIQ->Coffee_Shop). At this point we’d also want to ensure that coffee shops are only given the PlaceIQ->Coffee_Shop segmentation during certain times of the day (i.e. when they’re open).
Next, we’d make sure that any labels corresponding to a coffee shop, within the dataset, are attributed to PlaceIQ->Coffee_Shop. For example, Boston_OpenData_Retail could already have similar entries called Coffee_Shop & Cafe. These entries should map to PlaceIQ->Coffee_Shop, so we’ve built taxonomy connections to enable this to happen automatically.
PIQL then ingests user activities within these locations to answer questions about user behavior. For example, consumers seen visiting a Starbucks 6 times a week for the past two months are frequent visitors, and might be a more valuable – and reliable – audience to a marketer.
Implementation & Design
If the software engineer inside you wants to know how PIQL goes from a plan to a process, here it is (If not, skip down to “The Power of PIQL”). As mentioned earlier, PIQL was born out of the necessity to empower data analysts with a common high-level language designed to capture the essence of location-based reasoning. This rule-based, procedural language is made up of statements that are evaluated in the program order. Ultimately there are two types of rules: m-rules and b-rules. The former is used to express how physical locations and their attributes are combined; the latter is used to express user behavior or profiles as they interact with these locations .
To understand this design we can expand on the coffee shop example. In this case, we can add an m-rule parameter that differentiates between consumers who go to a coffee shop in a residential area, as opposed to a commercial area. Or, we could limit the rule to only include coffee shops that also serves beer & wine. Then, an example of a b-rule could be a modification of consumers to those who work no more than a 3 minute walk away from a Starbucks in Boston.
The above diagram shows the flow that takes a PIQL file and evaluates the rule. This flow is typical of how most programming languages work.
1. Parser – A file containing many PIQL rules is run through a parser that evaluates against a pre-defined formal grammar. The rules are converted to an abstract syntax tree that is later traversed in order to evaluate the rule.
2. Compiler – While the rules file is being parsed, the compiler ensures that rules are well formed (follow the correct syntax).
3. Typechecker – The typechecker ensures that rules aren’t trying to execute nonsensical instructions such as adding a number to a boolean.
4. Evaluator – Now that the rules have been compiled and typechecked, we can safely run the program and rely on it to produce the location-aware segments that we expect.
The Power of PIQL
The ability to rapidly segment based on consumer behavior is crucial to being able to respond to customer needs. This allows us to very quickly experiment with different behavioral models and tease out the underlying human behavior that the data indicates.
This level of granularity has enabled marketers to leverage location intelligence for a wide range of use cases, including media campaigns, real-world attribution, deep audience insights and strategic business decisions like retail site selection, foot traffic analysis and out of home attribution.