6 mins | 10 Jan, 2023

An introduction to the Pimcore Data Hub

Pimcore has had a data hub included since version 5. Initially, it was quite a basic feature, in more recent versions though the tool has seen some significant enhancements. Pimcores data hub is now much easier to configure and allows simple transfer of data between it and third-party systems. Following Pimcores general data management conventions, you have a lot of control over what's accessible and by who. This article will provide an overview of the Pimcore Data Hub and how to get started using it.

What is a data hub?

In the context of Product Information Management (PIM) or Master Data Management (MDM), a data hub facilitates the transfer of data from one system to another. It provides a single point of access to get data in or out of your platform. This is often used as a way to connect 2 systems together, like connecting a PIM to an eCommerce platform. It can also be used to look up data for use in another way, like creating a report, or populating a dashboard.

In the past, transferring data between independent systems was cumbersome and required specialist development skills. With the advancement of API-driven technologies, most modern digital technologies make access to data a simple process. A data hub is usually the access point which you can think of as the public gateway into your database.

The objectives of the Pimcore data hub

The main objective of the hub is to facilitate the transfer of data in and out of Pimcore. This could be object data, document data or asset data. The hub also provides access controls, so you can say who should and should not be able to push data to or pull data from Pimcore, and specifically which pieces of data they should be allowed to interact with.

Key concepts of using GraphQL in Pimcore

The API query language used for the Pimcore Data Hub (at this time of writing) is GraphQL. It is expected to allow other query languages to be used in the future, but for now, you write a query using GraphQL which allows you to access your data.

There are 2 key concepts to understand when using the tool that allow you to do different things with the data:

  • Queries - Look up data
  • Mutations - Change data before updating

Queries

Following Pimcore’s overarching conventions, there are specific ways to query documents, assets and objects.

Documents are typically web pages, snippets of a web page, PDF templates or links. These can also consist of a large number of smaller elements (Tables, AreaBlocks, Form fields, etc). It's possible to look up any item saved within a Pimcore document and fetch the content (data) stored within it. In addition, you can also query relations within your documents to understand related content and look up document properties. For multi-lingual uses, you may specify which translation of the document you want to fetch. There are a number of example queries available in the Pimcore documentation which is a useful starting point when first using the API. Document queries are most commonly used by those wanting to access or update content.

Assets are files stored within the DAM, typically images, PDFs, videos, and 3D CAD files. You can make assets available through the hub to offer those outside of Pimcore access to these files. This can be to share single files or folders. There are also sorting and filtering options available. Example asset queries.

Objects are where data is stored and will likely be the most common type of query you will look to make. Pimcore’s object data modelling is advanced and flexible, which means there are a large number of supported data types to query (there’s never been a piece of data we’ve been unable to retrieve, no matter how it's modelled). You can see the full list of supported data types here.

The Pimcore documentation is pretty good for understanding these principles and they provide a number of example queries to get you started. If those don’t help, there are also notes on how to create your own custom queries which I’d encourage you to investigate if you’re wanting to go deeper into what’s possible.

Mutations

Rather than just running standard query lookups, it's also possible to change data as you do so. As you might expect, you can mutate document, asset and object data (with the majority of each's respective elements supported).

For documents, there are 2 supported mutation modes available:

  • The ‘Free-form API’ will allow you to update a single document or 'editable'. This requires specific knowledge of the document you wish to update and could be considered quite an advanced option for Pimcore developers
  • The ‘Tree API’ uses a nested approach and might be more accessible to those just starting out.

For objects, the 3 supported DataObject Mutations are Create, Update & Delete. As you might expect though, this gives you a huge amount of freedom to manipulate data stored within your PIM and typically do whatever you might need to.

Query operators allow you to transform your data before its delivered, there are some useful operators available straight out of the box:

  • Date formater
  • Concatenaton
  • Element counters
  • Merging
  • Trim
  • Translate
  • Substring

It is also possible of course, to extend object mutations to add your own custom logic.

The GraphiQL Explorer

To make using the API easier for most users, there is a handy GUI called the GraphiQL Explorer bundled into Pimcore which allows you to configure your API request settings. You can create multiple configurations where you define your schema to use:

  • The Query Schema
  • Any Mutation Schemas
  • The types of data you will query (Documents, Assets, Objects)
  • The abilities you want available (Create, Read, Update, Delete)

There are also some additional settings where you can set permissions on who can use it and create your API key credentials.

The Pimcore GraphiQL Explorer GUI
The Pimcore GraphiQL Explorer GUI

Events

Events (based on Symfony’s event dispatcher) are triggered during queries and allow you to extend your functionality. Some example use cases for this suggested by Pimcore are:

  • Bypassing workspace permissions
  • Adding custom arguments
  • Adding custom query conditions
  • Enabling/Disabling responses
  • Effecting caching

Caching

It is possible to utilise caching for your API queries. This is especially useful for large or complex requests that require only periodic updating. When working with large datasets that have a lot of complex modelling to work through, API requests can not always be as fast as you might like. Using the output cache will help you here.

Localisation

Localisation allows you to change language when querying data. So, if you have a multi-lingual PIM you can allow access from different native speakers to the same content to read or write their own local translations.

Feature setup Overview

Getting up and running with your first API is actually pretty simple. Assuming you have a PIM setup with some data to access already, the data hub is ready to go following a few configuration steps:

  • Setup a new config
  • Configure your schema
  • Set security definitions
  • Add any additional settings

General setup

First, there are some general settings to go through. After creating a new configuration, you will give it a name and description and set it to be active or not. At this point, you can also add in any SQL conditions you want to include.

Pimcore Data Hub general setup
Pimcore Data Hub general setup

Next, you will configure the Schema settings. This is where you control what data should be accessible to your end-point. You can be very specific here, choosing an entity (object, document, asset) and the available fields - you can be very broad in nature or focus specifically on one entry stored within Pimcore.

In addition to the entity you are accessing, you can also choose which specific fields are available. Again, here you can be very specific with a wide range of options available.

Pimcore Data Hub schema setup
Pimcore Data Hub schema setup

Security settings provide more control over what those accessing the data can do. It's also where you define the credentials your API will use. You will need to:

  • Set your API key
  • Set permissions to Create, Read, Update or Delete (much the same as with Pimcores general permissions).
Pimcore Data Hub security setup
Pimcore Data Hub security setup

It's also possible to create custom permissions (although this is really for experienced Pimocre developers). Adding these will require some code to be written.

Once you have been through these steps, your API is ready to use and you can provide access to your database to the outside world. By default Pimcore provides a standard connection end-point:

(/pimcore-graphql-webservices/{configurationname}?apikey={yourApiKey})

It's possible to override this.

Who is the Pimcore data hub for?

The Pimcore Data hub is for anyone who wants to connect their PIM to another system or make data available to a third party. For developers, it's made API-driven data transfer much easier by removing the need to build custom bridges and providing a focal point (and setting some standards) for data-transfer activities. For admin users, the tools are now there to configure and launch APIs via a GUI.

This is still probably one of the more advanced features Pimcore has to offer so is really aimed at experienced Pimcore admins. The flexibility it provides though is great for anyone looking to be hands-on with the data stored in their PIM/CMS.

Conclusion

Modern digital platforms have benefited hugely from the development of API technology. It's made connecting different systems much easier and fostered a new approach to building composable architecture. Pimcore has always been at the forefront of data-driven product management and the data hub adds another tool to its suite of features to enable forward-thinking companies to open up their data sets in new ways. I hope this article has been useful in learning about the capabilities. If you’d like to go deeper or discuss how using Pimcore could improve your business operations, feel free to reach out to us.

Explore our services

Got a project in mind?