Things data protection / privacy (some AI), Sept/Oct 2024

GDPR Procedural Regulation: the Council seems to be progressing this, in October 2024.

CJEU cases: there have been several lately that others have covered, such as on commercial interests possibly being legitimate interests, so I won't for now. I just want to highlight a case from a few months back, which is relevant to employee policies and training/awareness-raising, and possible strict liability to pay compensation to data subjects, at least for infringements arising from employee action/inaction.

Adtech: IAB Tech Lab has launched, for public consultation, its PAIR protocol 1.0 for a "privacy-centric approach for advertisers and publishers to match and activate their first-party audiences for advertising use cases without relying on third-party cookies". Initially donated by Google, PAIR has been developed into "an open standard that enables interoperability between data clean rooms and allows all DSPs to adopt the protocol for enhanced privacy-safe audience targeting".

Equality, AI: The public sector equality duty and data protection, Sept 2024, UK EHRC guidance (with ICO input), including helpful examples of proxy data for protected characteristics under the UK Equality Act 2010, and a short section on proxy analysis of AI models, with a case study on the Dutch benefit fraud scandal that led to unlawful discriminatinon (from using biased predictive algorithms).

Open-source AI: from UK ICO's previously-asked questions, this Q&A was added recently even though currently the "Last updated" date indicates 11 April 2024.
Q: We want to develop a speech transcription service for use in our organisation, using an open-source artificial intelligence (AI) model. Can we do this even though we don’t have detailed information about how the model was trained? (see the answer! It seems call transcription is a popular use of AI, see other Q&A on that webpage on that topic, e.g. this and this. Also, compare a Danish SA decision from June 2024 on the use of AI to analyse recordings of phone calls.)

Oral disclosures?: talking of contrasting approaches, compare a Polish SA decision holding that oral disclosure of personal data during a press conference was not in breach of GDPR, whereas an Icelandic SA decision ruled that oral disclosures by police under the Law Enforcement Directive infringed that Directive.Yes, different laws, but they ought to be interpreted consistently. And I don't get how oral statements amount to "processing" wholly or partly by automated means under EU data protection laws, just as I don't get how there have been so many fines in the EU/UK regarding paper records without first holding that they form part of a "filing system" as defined.

ICO big PSNI fine: well-known by now (news release, MPN), but it underlines the point that the many surnames can be unique, and indicate religion and/or ethnicity (see Equality above on proxy data).

ICO: selected recent ICO disclosures, that the ICO decided to publish following FOI requests to it:

• How the ICO assesses incidents / possible personal data breaches: ICO internal guidance (request, PDB assessment methodology as of June 2023); seems to be based on ENISA's risk assessment for PDBs, which is unsurprising as that has been endorsed by both EDPB and ICO
• Territorial scope under UK GDPR, DPA 2018: ICO internal guidance (request, copy)
• What's a restricted transfer outside the UK: ICO internal guidance (request, copy); taking the outdated and misguided view that "transfer" is based on transfer of personal data's physical location, which is at odds with the ICO's own public guidance on transfers!
• How does ICO decide whether to publicise its intention to fine (request, emails on decision, more info)? This was on one concrete situation, but it's helpful to know the factors, again unsurprising, which I summarise below:
•  ICO default posture of transparency, although it considers each circumstance.
• This is consistent and fair with other similar cases where it has publicised the information at this stage.
• For deterrence regarding perceived central provider issues: "We are seeing a pattern of central providers having security issues with consequences for patients, publishing this will act as a learning/ deterrent for other processors with large central contracts, including the provisional fine will help clarify the seriousness of these issues".
• "The case has been extremely well reported and is well known, so this reduces the potential additional impact on the organisation and there is limited dispute about the facts of the attack."
• "Publishing the NOI [notice of intention to fine] and the provisional fine will help improve information rights practice and compliance among those we regulate."
• While it is possible that the fine value will change, as it is "provisional and subject to reps", this was balanced "the possible criticism of the ICO for changing the fine amount as the process concludes vs. the benefit of being transparent about the process... Idemonstrating that, if it does change, that is proof that the ICO does consider reps carefully and takes action based upon reps. This can serve to increase confidence in and awareness of our processes. I am comfortable that, subject to including suitable language to make clear it is provisional, that this risk is managed and the benefit is greater."
• "in this case, I have decided that publicity at this point allows for improved public protection from threat and hence is overridingly in the public interest. It is also already in the public domain."

DRCF: UK regulators the Digital Regulation Cooperation Forum are seeking input on their 2025/26 workplan by 8 Nov 2024. Unsurprisingly, the work includes AI, but also bilateral work on data protection and online safety, competition and data protection and illegal online financial promotions, and risks and opportunites of emerging technologies like digital identity, digital assets and synthetic media.

Data protection fee: The consultation on increasing the UK data protection fee has closed. The ICO's own response supported the increase, but didn't advocate for any change in the bases for charging the fee, although the government was open to views on that, so it seems there will just be an increase in fee levels but no substantive changes to the bases.

Dark patterns: while not limited to data protection, see OECD dark patterns on online shopping: countdown timers, hidden information, nagging, subscription traps, forced registration and privacy  intrusions, cancellation hurdles. Not dissimilar to the issues previously raised by UK regulators ICO and CMA on online choice architecture, control over personal data and harmful designs in digital markets.

Data transfers under the UN Digital Compact ("a comprehensive framework for global governance of digital technology and artificial intelligence"): the text is a bit vague and general on cross-border data flows, and 2030 is not exactly near-term!:

46. Cross-border data flows are a critical driver of the digital economy. We recognize the potential social, economic and development benefits of secure and trusted cross-border data flows, in particular for micro-, small and medium-sized enterprises. We will identify innovative, interoperable and inclusive mechanisms to enable data to flow with trust within and between countries to mutual benefit, while respecting relevant data protection and privacy safeguards and applicable legal frameworks (SDG 17).

47. We commit, by 2030, to advance consultations among all relevant stakeholders to better understand commonalities, complementarities, convergence and divergence between regulatory approaches on how to facilitate cross-border data flows with trust so as to develop publicly available knowledge and best practices (SDG 17)...

...We encourage the working group to report on its progress to the General Assembly, by no later than the eighty-first session, including on follow-up recommendations towards equitable and interoperable data governance arrangements, which may include fundamental principles of data governance at all levels as relevant for development; proposals to support interoperability between national, regional and international data systems; considerations of sharing the benefits of data; and options to facilitate safe, secure and trusted data flows, including cross-border data flows as relevant for development (all SDGs).

But on data protection more broadly, Objective 4. Advance responsible, equitable and interoperable data governance approaches, data privacy and security:

"We recognize that responsible and interoperable data governance is essential to advance development objectives, protect human rights, foster innovation and promote economic growth. The increasing collection, sharing and processing of data, including in artificial intelligence systems, may amplify risks in the absence of effective personal data protection and privacy norms...

...We commit, by 2030, to: (a) Draw on existing international and regional guidelines on the protection of privacy in the development of data governance frameworks (all SDGs); (b) Strengthen support to all countries to develop effective and interoperable national data governance frameworks (all SDGs); (c) Empower individuals and groups with the ability to consider, give and withdraw their consent to the use of their data and the ability to choose how those data are used, including through legally mandated protections for data privacy and intellectual property (SDGs 10 and 16); (d) Ensure that data collection, access, sharing, transfer, storage and processing practices are safe, secure and proportionate for necessary, explicit and legitimate purposes, in compliance with international law (all SDGs); (e) Develop skilled workforces capable of collecting, processing, analysing, storing and transferring data safely in ways that protect privacy (SDGs 8 and 9).

Survey on attitudes and awareness of emerging technologies, data protection, and digital products: There was a recent government survey of the UK public on the level of adoption and awareness of blockchain and immersive virtual worlds, attitudes towards pricing on digital platforms and behaviours regarding personal data control. But I can't yet find a summary of its outcomes, just the raw data.

Hungary: the Commission's decision to refer Hungary to the CJEU argues that Hungary's national law on the Defence of Sovereignty is in breach of EU law, including the e-Commerce Directive, the Services Directive, as well as EU Data protection legislation.

Canada: if attacker accesses and encrypts data without exfiltration for ransom purposes, that is still considered a breach that must be notified to affected individuals under Ontario’s Personal Health Information Protection Act (PHIPA), and the Child, Youth and Family Services Act (CYFSA).

Facial recognition & privacy / personal data: interesting and scary, students managed to adapt smart glasses to look up info on strangers in real-time, including parents' names!

(Also please see my blogs last week on security and AI: both have also been updated with more Sept links.)

GDPR compensation: strict liability? employee training / awareness

Case C‑741/21, GP v juris GmbH is not a recent judgment, but it still bugs me. Yes, it clarifies that mere infringement of GDPR provisions giving data subjects rights doesn't in itself necessarily constitute non-material damage, and that factors for determining fines, including when the same processing infringes multiple provisions, don't apply when determining damages for Art.82 compensation purposes.

However, what concerns me is this: the court also said, "it is not sufficient for the controller, in order to be exempted from liability under paragraph 3 of that article [Art.82], to claim that the damage in question was caused by the failure of a person acting under his or her authority, within the meaning of Article 29 of that regulation." And:

"...it cannot be sufficient for him or her to demonstrate that he or she had given instructions to persons acting under its authority, within the meaning of Article 29 of that regulation, and that one of those persons failed in his or her obligation to follow those instructions, with the result that that person contributed to the occurrence of the damage in question.

53      If it were accepted that the controller may be exempted from liability merely by relying on the failure of a person acting under his or her authority, that would undermine the effectiveness of the right to compensation enshrined in Article 82(1) of the GDPR, as the referring court noted, in essence, and would not be consistent with the objective of that regulation, which is to ensure a high level of protection for individuals with regard to the processing of their personal data."

Where should the line be drawn, then? It seems that, at least in the UK, a controller is not responsible for the acts of a rogue employee, who clearly becomes a controller in their own right. But if, despite an employer giving clear instructions to its employees, providing them with training, and implementing awareness-raising measures, a careless, mistaken or ignorant employee does something they shouldn't have (or doesn't do something they should have), and that results in the employer infringing GDPR, the employer is now still liable to compensate affected data subjects for the damage, including non-material damage, that they suffer arising from the infringement.

It had generally been thought that proving the organisation conducted training and awareness-raising measures would help it, at least perhaps in relation to potential fines for security breaches or the amount of fines, and some national regulators have taken post-breach training/awareness-raising measures into account there. Indeed, regulators generally consider that employee training/awareness measures are essential to comply with Art.32. However, it looks like such measures will not help employers to reduce or avoid compensation claims, at least under the EU GDPR.

Hopefully, given that regulators expect employee training/awareness-raising, this case won't result in organisations deciding to stop providing clear instructions/policies and training and awareness-raising measures for their employees, whether on security or other GDPR requirements. But, it doesn't exactly incentivise such measures... though it will certainly incentivise data subjects to claim compensation, including perhaps collective action lawsuits directly or through representatives, in cases where infringements were caused by the controller's employee(s) not following instructions or their training.  Proving that a controller "is not in any way responsible for the event giving rise to the damage" under Art.82(3) is a tough ask, but Art.82(3) says what it says. Effectively, this seems to create strict liability for compensation, unless the controller can disprove causation. Talk about rock and a hard place...

Things cyber security, summer / Sept 2024

Software acquisition: procurement teams acquiring third-party software may find useful NIST's list of questions (PDF) to ask and security considerations relevant before, during and after procurement; e.g. some of those questions could be included in contractual warranties and/or due diligence questionnaires. See also CISA's related Software Acquisition Guide for Government Enterprise Consumers: Software Assurance in the Cyber-Supply Chain Risk Management (C-SCRM) Lifecycle (PDF, spreadsheet), again useful for private sector organisations too.

Personal data breaches/PDBs: an SA is not required to fine/enforce for a PDB if that's "not appropriate, necessary or proportionate to remedy the shortcoming found and to ensure that that regulation is fully enforced" (Case C‑768/21, TR v Land Hessen).

Revised EU Product Liability Directive: the new EU Parliament has approved the text (Eur-Lex), so it just remains for the Council to adopt it (although Estonia is against the procedural rules); when published in the OJ thereafter, it will become law. Significance? For the purposes of no-fault liability for defective products, "product" will explicitly include software including that supplied via SaaS. Note the emphasis on safety and cyber vulnerabilities. 

Art.7(2): "In assessing the defectiveness of a product, all circumstances shall be taken into account, including... (f) relevant product safety requirements, including safety-relevant cybersecurity requirements..."

Also see the Recitals:"A product can also be found to be defective on account of its cybersecurity vulnerability, for example where the product does not fulfil safety-relevant cybersecurity requirements... relevant product safety requirements, including safety-relevant cybersecurity requirements, and interventions by competent authorities, such as issuing product recalls, or by economic operators themselves, should be taken into account in the assessment of defectiveness. Such interventions should, however, not in themselves create a presumption of defectiveness...The possibility for economic operators to avoid liability by proving that the defectiveness came into being after they placed the product on the market or put it into service should be restricted when a product’s defectiveness consists in the lack of software updates or upgrades necessary to address cybersecurity vulnerabilities and maintain the safety of the product... manufacturers should also not be exempted from liability for damage caused by their defective products when the defectiveness results from their failure to supply the software security updates or upgrades that are necessary to address those products’ vulnerabilities in response to evolving cybersecurity risks [unless not in their control e.g. owner fails to install it; yet, no obligation under this law to provide updates/upgrades but see CRA below]... a third party exploiting a cybersecurity vulnerability of a product. In the interests of consumer protection, where a product is defective, for example due to a vulnerability that makes the product less safe than the public at large is entitled to expect, the liability of the economic operator should not be reduced or disallowed as a result of such acts or omissions by a third party. However, it should be possible to reduce or disallow the economic operator’s liability where injured persons  themselves have negligently contributed to the cause of the damage, for example where the injured person negligently failed to install updates or upgrades provided by the economic operator that would have mitigated or avoided the damage."

EU Cyber Resilience Act (CRA) on "horizontal cybersecurity requirements for products with digital elements": the new EU Parliament has approved the text (Eur-Lex), so it just remains for the Council to adopt it; when published in the OJ thereafter, it will become law. Note, this aims to "set the boundary conditions for the development of secure products with digital elements by ensuring that hardware and software products are placed on the market with fewer vulnerabilities and that manufacturers take security seriously throughout a product’s lifecycle".

EU DORA Regulation, financial entities: there are corrections in the versions for FR, RO, SL [sic, SI?]

UK Cyber Security and Resilience Bill: while the UK recently designated data centres as Critical National Infrastructure (CNI), the CPNI list doesn't seem to have been updated accordingly yet. Note, this is not the same as extending the UK NIS Regulations to cover data centres (as the EU NIS2 Directive will do, though it's inapplicable in the UK post-Brexit). However, DSIT has indicated in its Sept newsletter (not on the gov.uk website yet) that the Bill will strengthen UK’s cyber resilience and ensure the critical infrastructure and essential services are more secure, by "strengthening the UK’s only cross-sector cyber legislation – the Network and Information Systems (NIS) Regulations 2018. Measures will include expanding the remit of the regulation to protect more digital services and supply chains". And just out: a DSIT webpage on this Bill. Currently it says little more about the Bill that what was in the King's Speech background PDF, but it does indicate that this Bill will be introduced to Parliament in 2025. (On ransomware under the Bill, please see below.)

Ransomware: in late 2023, Interpol and 50 countries including the UK signed a Counter Ransomware Initiative (CRI) joint statement on ransomware payments (US press release). The European Commission has now been authorised to negotiate, on behalf of the EU, the International Counter Ransomware Initiative 2024 Joint Statement (background on CRI). UPDATED: now see the full CRI guidance for organisations during ransomware incidents (news release).

(In May 2024, the UK NCSC with insurance industry bodies had issued Guidance for organisations considering payment in ransomware incidents, and the King's Speech detailed PDF in July 2024 stated that the forthcoming Cyber Security and Resilience Bill will be, among other things, "mandating increased incident reporting to give government better data on cyber attacks, including where a company has been held to ransom".)

UK communications providers & security: Ofcom updated its Network and Service Resilience Guidance for Communications Providers for telcos in early Sept 2024, following consultation.  Ofcom said, "Specifically, we are making clear that we expect them to: ensure networks are designed to avoid or reduce single points of failure; make sure key infrastructure points have automatic failover functionality built in, so traffic is immediately diverted to another device or site when equipment fails; and  set out the processes, tools and training that should be considered to support the requirements on resilience".  

Proposed EU CSAM Regulation: the Global Encryption Coalition is concerned about the Hungarian Presidency's 9 Sept 2024 compromise text, which would still require scanning of encrypted messaging services, undermining encryption and accordingly security and privacy. The Presidency is pushing for a partial general approach at the Council by as soon as 10 Oct 2024! (Good encryption FAQ).

Passwords: NIST's latest draft Digital Identity Guidelines: Authentication and Authenticator Management now states, among other things, that passwords:

• Minimum - "shall" be required to be 8 characters minimum, and "should" be required to be 15 characters minimum
• Maximum - "should" accept 64 characters (to enable passphrases)
• Types of characters - "should" accept ASCII, space, Unicode; but "shall" NOT require other composition rules like a mix of different character types - unlike what most organisations currently require!
• Change - "shall not" be required to be changed by users periodically (again unlike what too many organisations do), but change "shall" be required if there's evidence the "authenticator" was compromised (cf. that the password itself was compromised)
• No storage of password hints accessible to unauthenticated people (e.g. not logged in), and no prompts for knowledge-based authentication (like first pet's name) or security questions when choosing passwords

Payment webpages: fines have been imposed on companies under GDPR because their payment webpages got hacked, directly or indirectly, enabling criminals to capture customers' payment card details for fraud. The recent Frame Watch feature of ReportURI, helmed by noted security expert Scott Helme (if you'll forgive the pun!) alongside its existing Script Watch and Data Watch features, looks helpful to monitor and provide alerts for suspicious activity on payment pages.

Cloud forensics: post-data breach forensics on cloud services isn't easy. NIST's Cloud Computing Forensic Reference Architecture document, from July 2024, suggests ways to implement cloud architecture to faciliate forensics.

Aligning US federal agencies' cyber defence: CISA's priority areas aren't surprising: asset management, vulnerability management, defensible architecture, cyber supply chain risk management, and incident detection and response. The tricky bit is, of course, aligning systems/processes accordingly, e.g. by increasing operational visibility of assets, managing the attack surface of Internet-accessible assets, securing cloud applications etc., under its Federal Civilian Executive Branch (FCEB) Operational Cybersecurity Alignment (FOCAL) Plan. Again, much of this is of use to the private sector too.

Also of interest:

• The recently-published UK MoD's annual analysis of future global strategic trends 2024 mentions cyber and AI, unsurprisingly 
• NIST on Building a Cybersecurity and Privacy Learning Program
• US proposed rule intended "to Secure Connected Vehicle Supply Chains from Foreign Adversary Threats"
• US White House report, Roadmap to Enhancing Internet Routing Security 
• Many countries' agencies' Guidance on Detecting and Mitigating Active Directory Compromises (news release)
• Initiatives on closing the cyber skills gap (what about encouraging career changers, part-timers, retired people etc., not just incentivising teens/younger people into cyber careers?)
• UK NCSC on software bills of materials (SBOM)
• Dissection of exactly what went wrong leading to the 2023 UK emergency services outage - this sort of detailed sequencing of events, root causes etc. is always helpful for others to learn from
• Microsoft's Secure Future Initiative (for secure by design, secure by default, secure operations) - progress update and Sept 2024 report, initiated after attacks like the well-known successful attacks on Exchange for email. The SFI's 6 key pillars represent security 101 for all: protect identities and secrets, protect tenants and isolate production systems, protect networks, protect engineering systems, monitor and detect threats, accelerate response and remediation
• ADDED Cloud is now quite mainstream, but the cloud adoption checklist in US CISA's Considerations for Public Safety Cloud Computing Adoption from Jul 24 is generally useful for all organisations, even those not in public safety, and covers much more than just security issues 
• ADDED Hybrid cloud risks: Microsoft has noted threat actor Storm-0501 compromising on-prem environments then moving laterally "from on-premises to cloud environment, leading to data exfiltration, credential theft, tampering, persistent backdoor access, and ransomware deployment", targeting many US public and private sector organisations
• ADDED ENISA Threat Landscape 2024 identifies 7 prime cybersecurity threats, topped by threats against availability, then ransomware and threats against data, and analyses many publicly-reported incidents
• ADDED Canada: if attacker accesses and encrypts data without exfiltration e.g. for ransom purposes, that is still considered a breach that must be notified to affected individuals under Ontario’s Personal Health Information Protection Act (PHIPA), and the Child, Youth and Family Services Act (CYFSA).
• ADDED NIS2 Directive, very little time left before it's fully applicable! - draft national law implementations include:
• Ireland's National Cyber Security Bill 2024 (from 30 Aug 24)
• Italy

Things AI, Sept 2024

Open-source AI models: from ICO's previously-asked questions, this Q&A was added recently even though currently the "Last updated" date indicates 11 April 2024.
Q: We want to develop a speech transcription service for use in our organisation, using an open-source artificial intelligence (AI) model. Can we do this even though we don’t have detailed information about how the model was trained? (see the answer!)

AI Act: from Deloitte's AI Act Survey 2024, not many companies surveyed have started prep, nearly half feel partially/poorly prepared, over half think the Act constrains their innovation capabilities in AI, there were mixed views on legal certainty and on the Act's impact on trust in AI, and almost half thought the Act's more of a hindrance to AI-based applications! But, over a 100 companies have signed the Commission's voluntary AI Pledge under its AI Pact, that seeks to encourage organisations to implement AI Act measures ahead of its formal applicable dates.

Beyond the AI Act, see more generally:

• the European Commission's policy brief on Competition in Generative AI and Virtual Worlds, and 
• Europol's report on AI and policing and its benefits and challenges for law enforcement (summary).

Revised EU Product Liability Directive: the new EU Parliament has approved the text (Eur-Lex), so it just remains for the Council to adopt it (although Estonia is against the procedural rules); when published in the OJ thereafter it will become law. Significance? For the purposes of no-fault liability for defective products, "product" will explicitly include software including that supplied via SaaS. The text also mentions software as including AI systems. Also:

"A developer or producer of software, including AI system providers within [AI Act] should be treated as a manufacturer"... "Where a substantial modification is made through a software update or upgrade, or due to the continuous learning of an AI system, the substantially modified product should be considered to be made available on the market or put into service at the time that modification is actually made."

"National courts should presume the defectiveness of a product or the causal link between the damage and the defectiveness, or both, where, notwithstanding the defendant’s disclosure of information, it would be excessively difficult for the claimant, in particular due to the technical or scientific complexity of the case, to prove the defectiveness or the causal link, or both... Technical or scientific complexity should be determined by national courts on a case-by-case basis, taking into account various factors. Those factors should include...the complex nature of the causal link,  such as... a link that, in order to be proven, would require the claimant to explain the inner workings of an AI system...  ...in a claim concerning an AI system, the claimant should, for the court to decide that excessive difficulties exist, neither be required to explain the AI system’s specific characteristics nor how those characteristics make it harder to establish the causal link." 

EU Cyber Resilience Act (CRA) on "horizontal cybersecurity requirements for products with digital elements": the new EU Parliament has approved the text (Eur-Lex), so it just remains for the Council to adopt it; when published in the OJ thereafter, it will become law. Note, this aims to "set the boundary conditions for the development of secure products with digital elements by ensuring that hardware and software products are placed on the market with fewer vulnerabilities and that manufacturers take security seriously throughout a product’s lifecycle". Also note, "Products with digital elements classified as high-risk AI systems pursuant to Article 6 of [AI Act] which fall within the scope of this Regulation should comply with the essential cybersecurity requirements set out in this Regulation..." (see much more in Art.12 and Rec.51 which specifically cover high-risk AI systems, and Art.52(14)).  BTW, the Commission is inviting cybersecurity experts to apply to join its CRA Expert Group. Various criticisms of the CRA have been mentioned in my book/free companion PDF; here's another critique.

EU AI Liability Directive: added - Proposal for a directive on adapting non-contractual civil liability rules to artificial intelligence: Complementary impact assessment from the EPRS (as requested by a Europarl committee) "proposes that the AILD should extend its scope to include general-purpose and other 'high-impact AI systems', as well as software. It also discusses a mixed liability framework that balances fault-based and strict liability. Notably, the study recommends transitioning from an AI-focused directive to a software liability regulation, to prevent market fragmentation and enhance clarity across the EU" (PDF).

UK: the AI Act doesn't apply in the UK post-Brexit, so perhaps there are indeed more AI opportunities in the UK, on which Google has published a blog and fuller paper. The UK will make the AI Safety Institute (AISI) a statutory body as well as "identifying and realising the massive opportunities of AI" including for government/public services. (Here, the UK's not alone: a study for the European Commission emphasises AI's "significant potential" to improve EU public sector services.) AISI work includes assessing AI capabilities, e.g. Early Insights from Developing Question-Answer Evaluations for Frontier AI.

But, the GDPR still applies in the UK: ICO statement on LinkedIn's changes to its AI policy, so it is no longer training genAI models using UK users' data (opt-out link for others). There was separately an AI opt-out hoax that fooled a lot of people!

The recently-published UK MoD's annual analysis of future global strategic trends 2024 mentions cyber and AI, of course. UK civil servants (but not the rest of us!) are being offered free training AI-related courses, covering various aspects of AI, illustrating what's considered most important: Fundamentals, Understanding AI Ethics, The business value of AI, Gen AI Tools and Applications, Working with Large Language Models, Machine Learning and Deep Learning, Natural Language Processing and Speech Recognition, Computer Vision, and a Technical Curriculum.

Separately, case studies summarised based on the DSIT AI assurance techniques have been boosted by the addition of more products/platforms, on areas from governance, facial recognition e.g. for verification/identification, compliance management and bias assessment (even for NIST AI RMF, ISO, and NYC 144 bias audit with synthetic data!) to AI monitoring/audit. If you're planning to offer AI products to government (or beyond), it wouldn't be a bad idea to get your own products assured and listed similarly.

AI uses in the UK: a great use is autonomous robots to maintain fusion facilities. On health, a "novel ... AI tool, validated using NHS eye imaging datasets... could transform the efficiency of screening for Diabetic Retinopathy (DR)", while the MHRA  is calling for applications for manufacturers and developers of AI medical devices to join its AI Airlock regulatory sandbox; and, Reflections on building the AI and Digital Regulations Service. Added: AI platform via QR code for citizen science info on bathing water quality in Devon and Cornwall.

Collaboration on cybersecurity and AI research announced between the UK, US and Canada, to support defence and security

Equality, AI: The public sector equality duty and data protection, Sept 2024, UK EHRC guidance (with ICO input), including helpful examples of proxy data for protected characteristics under the UK Equality Act 2010, and a short section on proxy analysis of AI models, with a case study on the Dutch benefit fraud scandal that led to unlawful discriminatinon (from using biased predictive algorithms)

United Nations: much activity on AI, such as the final Governing AI for Humanity report on global AI governance, gaps, and international cooperation.

The recently (and almost simultaneously) promulgated UN Digital Compact is "a comprehensive framework for global governance of digital technology and artificial intelligence":

• Objectives agreed included: "Enhance international governance of artificial intelligence for the benefit of humanity"
• Principles agreed included: "Safe, secure and trustworthy emerging technologies, including artificial intelligence, offer new opportunities to turbocharge development. Our cooperation will advance a responsible, accountable, transparent and human-centric approach to the life cycle of digital and emerging technologies, which includes the pre-design, design, development, evaluation, testing, deployment, use, sale, procurement, operation and decommissioning stages, with effective human oversight"
• On Digital public goods and digital public infrastructure: "We recognize that digital public goods, which include open-source software, open data, open artificial intelligence models, open standards and open content that adhere to privacy and other applicable international laws, standards and best practices and do no harm, empower societies and individuals to direct digital technologies to their development needs and can facilitate digital cooperation and investment... ...We commit by, 2030, to: (a) Develop, disseminate and maintain, through multi-stakeholder cooperation, safe and secure open-source software, open data, open artificial intelligence models and open standards that benefit society as a whole (SDGs [Sustainable Development Goals] 8, 9 and 10)
• On  Objective 3. Foster an inclusive, open, safe and secure digital space that respects, protects and promotes human rights, they "urgently... Call on digital technology companies and developers to continue to develop solutions and publicly communicate actions to counter potential harms, including hate speech and discrimination, from artificial intelligence-enabled content. Such measures include incorporation of safeguards into artificial intelligence model training processes, identification of artificial intelligence-generated material, authenticity certification for content and origins, labelling, watermarking and other techniques (SDGs 10, 16 and 17).
• On Objective 4. Advance responsible, equitable and interoperable data governance approaches, data privacy and security, "We recognize that responsible and interoperable data governance is essential to advance development objectives, protect human rights, foster innovation and promote economic growth. The increasing collection, sharing and processing of data, including in artificial intelligence systems, may amplify risks in the absence of effectivepersonal data protection and privacy norms...
  ...We commit, by 2030, to: (a) Draw on existing international and regional guidelines on the protection of privacy in the development of data governance frameworks (all SDGs); (b) Strengthen support to all countries to develop effective and interoperable national data governance frameworks (all SDGs); (c) Empower individuals and groups with the ability to consider, give and withdraw their consent to the use of their data and the ability to choose how those data are used, including through legally mandated protections for data privacy and intellectual property (SDGs 10 and 16); (d) Ensure that data collection, access, sharing, transfer, storage and processing practices are safe, secure and proportionate for necessary, explicit and legitimate purposes, in compliance with international law (all SDGs); (e) Develop skilled workforces capable of collecting, processing, analysing, storing and transferring data safely in ways that protect privacy (SDGs 8 and 9)
• And Objective 5 was all about AI governance, not quoted in full here but
  "We will: (a) Assess the future directions and implications of artificial intelligence systems and promote scientific understanding (all SDGs); (b) Support interoperability and compatibility of artificial intelligence governance approaches through sharing best practices and promoting common understanding (all SDGs); (c) Help to build capacities, especially in developing countries, to access, develop, use and govern artificial intelligence systems and direct them towards the pursuit of sustainable development (all SDGs); (d) Promote transparency, accountability and robust human oversight of artificial intelligence systems in compliance with international law (all SDGs). (Also see UNESCO's consultation from Aug-Sept 2024  with a policy brief summarising emerging regulatory approaches to AI.)
• We therefore commit to: (a) Establish, within the United Nations, a multidisciplinary Independent International Scientific Panel on AI with balanced geographic representation to promote scientific understanding through evidence-based impact, risk and opportunity assessments, drawing on existing national, regional and international initiatives and research networks (SDG 17); (b) Initiate, within the United Nations, a Global Dialogue on AI Governance involving Governments and all relevant stakeholders which will take place in the margins of existing relevant United Nations conferences and meetings (SDG 17)."

US: global AI research agenda; proposed Reporting Requirements for the Development of Advanced Artificial Intelligence Models and Computing Clusters (i.e. cloud providers); "This includes reporting about developmental activities, cybersecurity measures, and outcomes from red-teaming efforts, which involve testing for dangerous capabilities like the ability to assist in cyberattacks or lower the barriers to entry for non-experts to develop chemical, biological, radiological, or nuclear weapons." One I missed earlier: the IAF's paper on Risk/Data Protection Assessment (for AI) as Required by U.S. State Privacy Laws.

US FTC: action against "multiple companies that have relied on artificial intelligence as a way to supercharge deceptive or unfair conduct that harms consumers... include actions against a company promoting an AI tool that enabled its customers to create fake reviews, a company claiming to sell “AI Lawyer” services, and multiple companies claiming that they could use AI to help consumers make money through online storefronts." 

And some miscellaneous things...

Hallucination issues with LLMs remain: a recent egregious example.

Comparing chatbots: interesting open-source tool to compare different (anonymized) chatbots by asking them the same questions, and do choose the best answer. See its leaderboard, currently OpenAI's o1-preview is top!

Cognitive bias: humans tend to think fluent content (e.g. LLM-generated) is more truthful/useful than less fluent content, which can produce systematic errors.  Of course, this tendency is why even hallucinationary genAI output can be trusted and believed by humans!AWS scientists argue that "human evaluation of generative large language models (LLMs) should be a multidisciplinary undertaking that draws upon insights from disciplines such as user experience research and human behavioral psychology".

AI users: apparently have a healthier relationship with work than colleagues who don't use AI! Although of course AI has been the reason for some job cuts.

Interesting article on AI hype and another on the importance of human thought and judgment when using AI.

ADDED:

• D&TA Data Provenance Standards v1.0.0 (GitHub) from the Data & Trust Alliance (exec overview), partnering with OASIS. These are billed as "the first cross-industry metadata standards to bring transparency to the origin of datasets used for both traditional data and AI applications": use case scenarios, online metadata generator. Supporters include (from the tech sector) IBM, Meta and Salesforce, but also many from the manufacturing industry. (See also C2PA, Coalition for Content Provenance & Authenticity, on "An open technical standard providing publishers, creators, and consumers the ability to trace the origin of different types of media").

• Facial recognition & privacy / personal data: interesting and scary, students managed to adapt smart glasses to look up info on strangers in real-time, including parents' names!

(Also see my separate blogs on privacy / data protection and on security: links now added.)

Browser cookie settings & consumer preferences - UK study

"Evaluating browser-based cookie setting options to help the UK public optimise online privacy behaviours" (PDF), a study for the UK Department for Science, Innovation and Technology on consumer preferences conducted  between Aug and Dec 2023, concluded that:

"...We recommend that any future cookie setting option should be interactive and detailed to a sufficient level that participants understand the real-world impact of accepting or declining a number of different options, e.g. that ‘functional’ cookies include login details, website preferences (language, currency), see Appendix 2, Figure 5. These setting designs secure stronger engagement by breaking participants out of the habit of automatically accepting all cookies purely for the sake of expedient access to the browser; furthermore, participants are satisfied after such a process of critical engagement...

...People remain divided over the idea of browser-based cookies. To improve sentiment, any future browser-based cookie settings should include features that will enhance web users’ feelings of control over their data (e.g. frequent prompts for updates, options to adapt preferences by types of websites, or for specific websites).

Participant engagement and satisfaction improved when they had access to more functionality details, an interactive interface to select their preferences, and timely prompting about privacy. As a result, should browser based cookie management systems replace the website level settings, we recommend that browser-based cookie setting design should attempt to disrupt users’ habits of automatically accepting through novel designs to create a dissonance with what they are used to seeing. Furthermore, any cookie settings that encourage participants to make a privacy-protective choice will lead to higher satisfaction regardless of initial preferences."

But if there's too much detail in the expanded info, users may just ignore them. And I'm not so sure about frequent prompts to users... that doesn't provide a great user experience. Strictly, when users have accepted cookies for a site, they should probably be told about their right to withdraw consent to cookies everytime they return to the site, but that doesn't really happen, at least not in a "disruptive" way, presumably because that's not great for UX too (popups continuing to appear even if you've accepted cookies previously?!).

AI and GPAI developments/info, Sept 2024

Some AI-related links, which I hope will be of use:

✨Final text of AI Pact pledges, promulgated by the European Commission to get tech companies to comply voluntarily with (at least some of) the AI Act before its formal applicable date. So, the text is not dissimilar from that of the EU AI Act, and indeed the G7 Principles from the Hiroshima Process

✨Human oversight of ADM / automated decision-making: videos from the EDPS's workshop, 3 Sept 24

✨AI Act briefing for European Parliament, 2 Sept 24

✨BSA (The Software Alliance) Best Practices for Information Sharing Along the General Purpose AI Value Chain. There's some overlap with the EU AI Act's GPAI requirements, 3 Sept 24

✨Computer & Communications Industry Association's recommendations on GPAI code of practice, Aug 24

✨Note that the Council of Europe's AI Treaty, signed by the UK, EU and others, will come into force only on the first day of the month following the 3-month period after 5 signatories, including at least 3 Council of Europe member states, have ratified it. (On treaties/conventions, see the differences between signing versus ratification versus accession).

To support the Treaty's implementation, the COE's HUDERIA is a "legally non-binding methodology" for Risk & Impact Assessment of AI Systems for Human Rights, Democracy and Rule of Law (good summary). The UK's Alan Turing Institute is assisting on HUDERIA. The COE Committee on AI is considering HUDERIA soon. The European Commission & Council of the European Union are also involved. What's the betting as to how much HUDERIA will influence what is going to be required in Fundamental Rights Assessments for certain high-risk AI systems under the AI Act?

(Compare the US Department of State's risk management profile for AI and human rights, July 2024)

✨Gartner Predicts 30% of Generative AI Projects Will Be Abandoned After Proof of Concept By End of 2025. Certainly, genai risk disclosures in US SEC filings are increasing. 

✨A data scientist has written a great outline of a practical approach in the face of the pressure to to AI-ify everything ASAP. Consider, is using AI always the best solution? Especially given that AI systems will increasingly be subject to more onerous obligations than non-AI systems (e.g. under the AI Act), is it always best to use AI when non-AI approaches/methods could work equally well or perhaps better?

✨What's up with the planned AI Liability Directive? It won't automatically be brought into force, but  a June Parliamentary Committee report stated that "the legislative work under the leadership of the Committee on Legal Affairs... will continue under the new Parliament. In the meantime, the Committee... requested an additional impact assessment from the European Parliament Research Service (EPRS). It is to primarily deal with the compatibility of the three legal acts mentioned [AIAct, product liability etc] and the risk-based concept and is expected to be completed by the end of 2024". Note the date for the additional impact assessment - end of 2024. So it's very unlikely this Directive will be passed this year, perhaps ever.

(I had tried to post the above on LinkedIn last Thursday, please see this AI developments post, but LinkedIn saw fit to demote it and not show it in people's feeds, so I think it's best to blog here instead, and at least there's space for me to flesh things out a bit more here.)

Are AI principles like standards?

 AI principles, indeed AI standards, seem to be ever proliferating...