LEVEL OF AGREEMENT

MOST DISCUSSED PARAGRAPHS

P5 3.Stresses that any future regulation should
6 6
P21 14. ... 
4 0
P18 12.Calls for the designation by each Member
3 8
P4 2.Notes that the framework should apply to a
2 7
P10 6.Recalls the importance of ensuring the ava
2 7
P11 7.Stresses that where public money contribut
2 6
P8 Consumer protection: transparency and explai
1 11
P9 5.Believes that consumers should be adequate
1 10
P13 8.Underlines the importance of ensuring that
1 8
P17 11.Notes that it is essential for the softwa
1 9

LATEST COMMENTS

I would add that there is a need to model operational accountability and impact assessment so as to deter public and private entities from implementing AI systems frivolously (as in Sweden, where “the Swedish data protection regulator recently banned facial recognition in schools, based on the principle of data minimization, which requires that the minimum amount of sensitive data should be collected to fulfill a defined purpose. Collecting facial data in schools that could be used for a number of unforeseen purposes, it ruled, was neither necessary nor a proportionate way to register attendance.”) I would want to empower the people who may be adversely affected by AI, by recommending mandatory creation of simple, transparent, time-boxed mitigation channels such that humans can easily request human review of AI decisions that may have been made in error. See CiGi summary here: “Policies around AI systems must focus on ensuring that those directly impacted have a meaningful say in whether these systems are used at all, and in whose interest.” The report also references: “The Algorithmic Accountability Act of 2019, a proposed bill in the United States Congress, attempts to regulate AI with an accountability framework. This legislation requires companies to evaluate the privacy and security of consumer data as well as the social impact of their technology, and includes specific requirements to assess discrimination, bias, fairness and safety. ***Similarly, the Canadian Treasury Board’s 2019 Directive on Automated Decision Making requires federal government agencies to conduct an algorithmic impact assessment of any AI tools they use. The assessment process includes ongoing testing and monitoring for “unintended data biases and other factors that may unfairly impact the outcomes.”**** URL - Centre for International Government Innovation -https://www.cigionline.org/articles/artificial-intelligence-policies-must-focus-impact-and-accountability
“...meaningfully contested with a human re-evaluation within a specified time frame.”
Dear Alexandra, dear team, all your suggestions are highly valuable, thank you very much for this initial report! For my MA thesis, I currently analyse biometric and emotional AI. A point that struck me was the absence of data protection for emotional datasets (eg. in voice assistants, FRT or sensors) in form of pseudonymous aggregate data by the GDPR. However, I'm not sure in how far the ePrivacy directive would/will tackle this issue. Below I include a paragraph from my draft (!) thesis to illustrate the point: "The large-scale data collection of information on individual’s emotional states are the most concerning developments in relation to biometric and emotional AI technology. In fact, privacy protection of individuals is not always granted. Information about emotional states, gathered with biometric technology, can be highly valuable even if individuals cannot be singled out. Especially aggregate datasets on emotional behaviour can be gathered without containing ‘personal’ or ‘sensitive’ information. Let us consider the case of a surveillance camera in a public space: The video material collected by the camera could be analysed by an algorithm in order to ‘read’ facial expressions and detect emotions by people. This example triggers a range of issues. First, it is not clear whether the data was given by consent. Second, although the information about emotions is surely rather personal information, because the camera cannot make links to an individual, their emotions are not considered ‘personal’. Third, if a certain person would be regularly filmed, and its emotions were to be tracked frequently, its safety, consumer and/or fundamental rights are at stake. The tracking of facial recognition emotions is thus a particularly critical use case of AI technology, based on large-scale datasets and algorithmic infrastructures in combination with biometric technological artefacts." The main idea/source is Andrew McStay's book "Emotional AI: The Rise of Empathic Media" (2018). Any questions, don't hesitate to contact me. Wishing you the best of success with the report! All the best, Rosanna
'for example via APIs'
consumer and other fundamental rights
ex ante human rights impact assessment should factor in the safeguards which should apply. Broader question before the risk-based question is that are certain applications/ uses completely in violation of fundamental rights therefore not legal. JURIs original report does not cover this
It would be nice to have specific feedback from you on the suggestions/paragraphs above as I don´t assume that you are against establishing an ethical foundation for AI. Correct?
Beyond open data requirements, I would add open procurement and open contracting standards and mandatory disclosure of human rights impact assessments
Do not create an "AI authority" with regulatory powers either on the MS or EU level. That should happen within existing enforcement bodies with an additional level of coordination. To support them with resources and expertise, create independent centres of expertise ​​on AI​ on a national level to monitor, assess,conduct research, report on, and provide advice to government and industry in coordination with regulators, civil society, and academia about the human rights and societal implications of the use of algorithms, automated decision-making systems, or AI.
Agreed that bot private and public sector should ensure a high level of effective remedies. Just noting that there are areas where the public sector obligations are absolute and the public sector is also responsible to ensure that the robustness of remedies as described is enforced in the private sector as well.
I would just add here that 'non-personal' data, and models trained on it, could be repurposed for harmful purposes, so there should nevertheless be some oversight of how models could be reused.
Access Now is not in favor of a high/low risk approach based on self-assessment. In areas where individual or collective rights are impacted the objective should be to avoid harms and violations and not to mitigate them.
With the changes Daniel suggested I'm supportive of the list. Any horizontal measure, however, should focus on the decisionmaking process and not a set of technology.
you could specifically mention rule-based systems, as they can have equally serious effects as machine learning based systems. I would suggest including it in the list after machine learning. I also think there is no need to mention deep learning specifically, as it's covered under machine learning.
Would such an authority be triggered by consumers directly? Or by public interest? Or would it investigate all systems that are deployed by specific stakeholders?
This must include artefacts of the risk assessment process outlined in P5.
[...]; the risk assessment of a specific system must be subject to regular re-evaluation;
Is this meant to capture the entire set of systems that may influence human decision making?
It's difficult to imagine which systems this could possibly apply to. Systems that process synthetic data only, i.e., research prototypes before evaluation?
Dear Alexandra, First of all, great work so far. I really appreciate your approach to ensure the high level of consumer protection. However, to reach this goal, I recommend NOT to differentiate between consumers and professionals. The reason for this can be seen in the case-law of consumer law/unfair completion law. Usually, consumers do not enforce their consumer rights as the legal process might be complicated and expensive. Therefore the level of compliance is low. In contrast, cases in which the rules apply to all subjects (consumers and professionals) show that the companies addressed by the provisions comply more likely to these rules. The mere reason for this is that the risk of seeing suit is much higher. To sum up, to ensure a high level of consumer protection it is necessary to create a recognizable legal risk. The best way to do so is to set companies against each other. From the perspective of the addressed companies (user of algorithms), this proposal is also advantageous because the implication would be much more manageable. Moreover, SMEs as such are also worthy of protecting. Imagine, for instance, a small Cafe which is discriminated by the Google algorithms. There would be no chance of defence. Greetings from Berlin Johannes Stuve (LAG Digitales Berlin/BAG Digitales)
It would be useful to give examples of existing authorities which could be designated (eg, Data Protection Authorities?)
We would delete this part. Also given that it is still unclear how to identify the level of risk, we are concerned that activities that may follow under 'low risk' would require additional obligations.
I only found out about all this from a Facebook post of your article/report. I am an American political scientist/futurist and a big fan of the EU and opposed Brexit. Howevre, this is probably the kind of legislation that the Tories used to get Brits to want out of the EU. While most EU regulatory regimes are superior to American ones, this particular "advance" has been one I've worried about since I became somewhat well known by an article wrote in The Futurist in 1981 called "Teledemocracy." I recall a conversation I had with the late great futurist Alvin Toffler on this very subject. It was during our watching Kubrick's 2001 and the subject was AI, although that term had not yet been invented. You may know or recall that in that motion picture, the mission was programmed so that "Hal" the computer was in charge. His program had been altered but the crew was unaware of the alteration in the programming. It took human ingenuity to finally get "hal" to obey the human crew before disaster ended the mission. So, the question was: Can you trust AI without allowing for humans to make the final decision when human life is effected? This was the futurists question as old as Socrates' question: "Who will guard the guardians?" The answer, my friend, is blowing in the wind, and this report is the wind, not the answer. As Alvin Toffler told me then, and it is particularly applicable now, "Experts serve humans, not humans serve the experts." The same must be the mantra applicable to AI. AI cannot be allowed to make any such decisions. It is only to present an orderly set of options to human "reprsentatives" of the people. What you are headed for is the worst dystopian "governement" of the machines, by the machines an for the machines. Stop with this madness. There is no way to program empathy into AI. Withhout it, all decisions made by AI are "artificial" by definition. If the EU passes such a law or regulation, then whatever country I lived in within the EU, I'd ask for an immediate "Exit." Dr. Ted Becker

MOST ACTIVE USERS

	JT	5	12

fannyh

5

3

Sarah EDRi

3

1

dleufer

2

9

	ErnaniCerasaro - BEUC	2	9

Nakeema

2

2

jgfna

1

13

becketl

1

5

	JohannesStuve	1	1

rosannafanni

1

0

Mari

0

20

LeifTree

0

20

	Verena	0	20

StefanieK

0

20

geekygekko

0

20

aemf

0

19

	njain	0	4