Dark patterns, dark algorithms, dark user experiences, what is that? You might ask. I will define them as the following for this piece:
Dark pattern - anything designed for malintention
Dark algorithm - an algorithm that is designed as a dark pattern
Dark UX - a user flow on the internet that is designed as a dark pattern
All of these patterns are designed intentionally, or if not, are the act of serious neglect. It was also super hard to aggregate these as recurring issues because there is no central place to discuss these issues and their prevelance, since sickness is taboo.
Ok cool, so how do these manifest in health insurance, you might ask?
Another reminder that businesses and their owners are inclined, or even at times duty-bound, to maximise the earnings for the stakeholders. Every day, the boundaries on what should be perceived as morally corrupt just get pushed farther.
It’s a troubling trend then that we are training the machines to do the same for us — imparting intelligence to maximise benefits for organisations who build these AI systems. A fascinating thread this that captures the fallouts of such training purely geared towards such optimisations.
Almost 10 years ago I had a prototype where I played around with using a simple genetic algorithm to breed enemies that would be good against whatever play style you chose. The fitness function in it was just how long they could stay alive against you
— Tyler Glaiel (@TylerGlaiel) January 18, 2018
if you optimize games for money you get mobile lootbox garbage. if you optimize videos for watch time you get vlogs that are 95% filler. if you optimize social media posts for view counts you get clickbait, lies, and freebooting
— Tyler Glaiel (@TylerGlaiel) January 18, 2018
AI is really good at finding local minimums and maximums. Like "oh, promoting 45 minute videos gets more watch time than 5 minute videos". Its terrible at recognizing that promoting that kind of garbage will eventually cause people to leave for better platforms
— Tyler Glaiel (@TylerGlaiel) January 18, 2018