Many platforms that process personal data such as HR systems, case-management tools, CRM platforms, ticketing systems or compliance communication channels are making AI-powered features available to customers to facilitate their tasks and support them with their jobs. Typical functions that are provided as an “AI add-on” to the data management platforms are: summarization, automatic classification, translation, chatbot-style analytics. Those applications are normally optional for the customers and are in most cases, subject to a separate fee.
But are those AI functions “innocent” or should we consider compliance implications if the platform processes personal data?
Of course, if the platform processes personal data and the AI add-on functionality involves the same, there are different aspects that should be considered by a data controller that wants to activate such AI supporting features. The main questions to ask oneself are: does activating this feature change what we are actually doing with personal data and how data are secured considering the adoption of the new feature? This is probably the core reasoning to start the evaluation.
The First Point: Responsibility of Processing
Where the organization is a controller and the platform provider a processor, that allocation of roles does not change because the vendor adds a feature. The controller’s accountability remains the same and the obligation of the processor to act only on the controller's documented instructions, also still applies. In fact, activating a new (AI-powered or not) feature would still qualify as instruction that the controller gives to the processor to perform operations on personal data. Before the AI feature is even activated, the customer decides what data goes into the system and, by activating a new AI feature, it still must be able to justify that choice including demonstrating lawfulness of processing.
In practice, when assessing the compliance of an AI add-on to a platform the controller should keep in mind that it is still the primary responsible entity towards data subjects and verify what the vendor has put in place from a security and contractual standpoint.
Applied Security Measures and Contractual Safeguards
Solution providers normally provide documentation on the use of AI, set out the security principles and rules applicable to their AI features and offer AI-related contractual addenda to supplement existing data processing agreements in order to cover the specific scope. From a branding perspective, the documentation, dedicated webpages and information materials from the vendors, may seem reliable however, from compliance perspective, it is important to make sure that the presented content has consistency and check the critical items for the assessment of the validity and reliability of this documentation. We want to give a high level overview of those items and highlight which are the ones to be addressed first.
Key contractual and technical safeguards should include a clear prohibition on using customer data to train or fine-tune AI models, appropriate data residency controls, and limits on data retention during inference. For EU-originating personal data, EU-based processing can reduce exposure to international transfer requirements under Chapter V GDPR. Organizations should also verify how prompts and other input data are deleted after generating an output or retained by the provider and ensure that standard security measures such as role-based access controls, encryption, and logging are implemented in line with Article 32 GDPR. From a contractual perspective, on the other hand, it is critical that the obligations are stated in writing and made bound by appropriate agreements. This means that any AI addendum to the DPA should be regularly signed by the parties and any specific details (such as retention period or data hosting) is specified there. Of course, in the addendum to the agreement, a complete list of subprocessor should be included, together with the transfers mechanism, if required. This is very important because it is very likely that the vendor will use the services of an AI model developer or an AI system provider.
The Risk of Personal Data Processing
On top of the security review, the other critical point is related to the core processing of personal data. As a first step, we should look at what type of data categories are processed: if special categories of data are processed in the platform, more attention should be paid to the AI features and related personal data processing. Secondly, we should look at what is the impact of the AI feature on personal data use and purposes of processing. As it was mentioned before, the question to ask is: does the AI feature change the purposes for which personal data were collected and processed in the first place? This of course, depends on the goal of the AI feature and on the outcome deriving from it.
A practical way to assess AI features is to distinguish between administrative support, analytics, and genuinely new uses of data. Low-risk functions such as summarization, translation, rewriting, or human-confirmed classification will often remain within the original processing purpose. Analytics that combine multiple records to identify trends or generate broader insights may constitute further processing and should be assessed by a compatibility purpose test, as required by GDPR. This is particularly important where special categories of data are involved. In fact, it is well known that those categories require specific legal bases for processing and should be subject to stricter security safeguards. AI features outcomes (and previous processing) that are unrelated to the original purpose, especially for example training AI models on personal data, require separate justification and potentially a new legal basis. Anonymization or robust aggregation can materially reduce these risks and, where data is genuinely anonymous, take the processing outside the GDPR's scope.
Despite the level of risk and on the final outcome/purpose of the AI feature, a mandatory step must be that human review takes place as a default obligatory process before any AI-generated output is relied on or acted upon. This means that any output that is generated by the AI feature (being it a summary or a list or a translation) should not be used as it is but must go under accurate human review. Besides accuracy issues, failing this step may imply major concerns related to automated decision-making processes (read about a recent fine due to automated decision-making processes here).
Before Switching Anything On
As a conclusion, the main key takeaways to keep in mind before deciding if the activation of an AI powered feature offered on a platform processing personal are related to some essential points:
- security measures and contractual obligations (as a backbone of the processing in general)
- outcome of the feature and purpose of the processing by the AI add-on
- human review as a mandatory step
Depending on the type of feature and data hosted on the platform, the criteria of assessment of course vary. However, it is key to ensure compliance at any step of the data processing, especially in consideration of the use of new technologies involving different parties.
No comments