v2.3.0 Release: New columns and Apple News Real-Time data

We're making improvements to the Parse.ly Data Pipeline! We listened to your feedback and have incorporated it into release 2.3.0, scheduled to go live in the next month.

There are parts of this new version that are not backwards compatible and could cause breaking changes depending on how you have your queries configured. ****To prepare for the release and update your queries ****please see all details in our release post.

Prepare for the release

What's changing?

We will be adding new data as well as new columns, so you will likely see an increase in the file sizes and number of rows. The updated schema documentation is available here.

Changed columns:

• flags_is_amp: This was an undocumented field that has existed for a few years. However it was typically populated as null, instead of false. This will now be of type Boolean and either true or false.

Additional columns:

• Any column that does not appear for an event will receive a null value. This logic will look at the schema that we have defined and auto-populate missing columns with null. All columns, regardless of whether they are null or not, will now be available for every event. This means that each event will have 122 columns, with null values populated where appropriate.
• channel: The Parse.ly-defined channel the event came in on. Can be strings like fbia (Facebook Instant Articles), apln-rta (Apple News), amp (Accelerated Mobile Pages), or website. If we add a new channel, that value will appear here. (Note that your Parse.ly account must be integrated with AMP, Facebook Instant, or Apple News for those values to appear.)
• pageview_id: A unique identifier for the pageview associated with an event. This will remain consistent for all events for a given pageview. This allows for sophisticated aggregation in your data warehouse such as correlating all heartbeat events for each pageview event. This will either be null, or a long integer like 17542680 and will always appear.
• pageload_id: A unique identifier for the pageload of an event. This is useful for single-page apps where there may be multiple calls to trackPageview for a single page load. This will either be null, or a long integer like 17542680 and will always appear.
• videostart_id: A unique identifier for a given videostart event, allowing you to correlate other events, like vheartbeat to their originating videostart. This will either be null, or a long integer like 17542680 and will always appear.
• schema_version: This is a new field and indicates the matching schema version from pypi and the parsely_raw_data repo. For example, this new release will be 2.3.0. The other versions that are on pypi can be found here: https://pypi.org/project/parsely_raw_data/ We will also have our documentation by schema version, enabling back-referencing of older versions for historical data.

Additional data:

1. Parse.ly now supports a real-time Apple News integration, available in the dashboard and the data pipeline. Please reach out to your account manager or reply to this email for more info.

   1. Once your Apple News integration is complete, you can expect the following:

      1. apln-rta will appear as a value in the new channel column.
      2. For the Apple News events, you’ll also find some of the original data from Apple in the extra_data object under the apln key. This is a passthrough of what we receive from the Apple News API and contains fields specific to that platform. For more details on the values in extra_data and how to filter for Apple News Real-Time Data, see our post here.

   2. If using SQL with Parse.ly data, you'll likely want to GROUP BY the channel field to do metric calculations. This is because certain metrics—like unique visitor counts—only make sense on single channels.