Old met new at Huge inc. recently in Dumbo as millennials and creative professional came face to face with technology that could on day do their job. The panel entitled The Future of Curation; Artificial Intelligence and Creativity was moderated by Huge inc’s own senior interaction designer, Jenny Clark. Along with her to explore the topic were Ramzi Rizk, CTO and founder of Berlin based artificial intelligence image curation and classification startup EyeEm. Providing the contemporary take on curation were Josh Raab , photo editor at TIME and Kamelia Aryafar, senior data scientist at Etsy.
Josh gave the audience a taste for what media curation process looks like now from a publisher’s perspective. Contrary to popular conception, most major magazines and digital publications don’t spend a lot of time in Photoshop crafting the perfect picture to complement a story. Rather, photo editors like Josh receive thousands of images a day through the press wires from field reporters, independent photojournalists and other media organizations. The job is to select which photos fit the content of the article and will elicit the strongest response.
However, what makes a good photo is in the eye of the beholder; human or algorithm. Josh described an experiment at TIME to test how photos were eliciting responses from these different groups. “We took the top 20 photos by likes from the TIME Instagram feed and had both human photo editors rank them as well as EyeEm’s AI algorithm”, which provides an image score from 1 to 100. The results couldn’t have been more different.
“While the Instagram audience loved celebrities like Beyoncé and fuzzy animals, the human editors looked for things like image composition and lighting while the algorithm seemed rank based on features like bright colors and high contrast.” The aesthetic gap was telling from both a human and technologic perspective and challenged the definition of a good photo. Is the value of a photo based on its objective quality or its content and the emotion it elicits? To further drive home the point, Josh ran the now famous picture of the drown Syrian refugee child Alan Kurdi through EyeEm’s system and it only scored a 55. If an editorial decision was being made on score alone, the now iconic photos would have been glossed over.
Despite AI’s subjective limitations, Josh acknowledged that it will change the future of the photo editor’s job. “In some cases, AI can be helpful, lots of images come in. We used to have professional who went through every image that came in every day for their whole career … you just can’t do that anymore.” While acknowledging AIs usefulness as a screen, Josh still emphasized its limits “The AI can narrow it down, but it doesn’t know me well enough. If it could learn from my boss’, or boss’ boss’ selections that would be great.”
Approaching curation from an ecommerce perspective at Etsy, Kamelia was far more optimistic. “Our objective function is to make a large marketplace feel smaller for you” she explained, and went on to elaborate how, and why, AI was perfect for the task. “Before, we didn’t have the data sets of appropriate size. now we have finally reached a point we have so much data we can get personalized signals not possible before.” Kamelia went on to explain why the ability of AI to work with image data was so important for the future of curation. “Personalization used to only use text, now we can use images as signal for personalization, we can create more coverage and get more accurate information” However, Kamelia also emphasized that not all decisions were algorithmic. “we have blogs, editorial teams and featured listings based on a curator’s choice. It’s not all one or other, it has to be a collaboration”
Although many new opportunities were highlighted from the application of AI to these datasets, new source of anxiety also crept in. Aaron Marks, co-founder of fine arts and culture subscription service CulturePass, posed one such concern. “Data sets are backward looking; will AI funnel us into one color or palette? How do you introduce element of surprise or chaos?” Jenny also steered the conversation towards data integrity “How do you make sure you have a diverse training set of data, that it’s not just all from white males?” Ramzi conceded “We have a problem with bias in machine learning.” He continued “Artificial intelligence is a misnomer, it’s not intelligence, its statistics” Regarding the algorithm “It sees 100 photos you say are good and 100 you say are bad and you get a classifier. It’s hard to look under the hood and extract features its scoring on.”
“Artificial intelligence is a misnomer, it’s not intelligence, its statistics” – Ramzi Rizk
Having acknowledged the potential biases and shortcomings of AI, the panel discussed way to circumvent some of these issues. Regarding the need for diverse training set data, Ramzi highlighted the user generated nature of the photos on EyeEm “We have men and women contributing, young photographers and old, from all the continents and background you can think of.” Kamelia echoed those sentiments at Etsy, “For us, diversity and discover is a very big topic, we try a lot of thing to add diversity as an element of surprise.” She even explained a solution involving AI, “I can find all items similar to your taste and score each item. Then I can purposefully take poor scoring items and introduce them to show you something new you might not yet like.”
Ramzi also emphasis the use of a sliding lookback window on data to help understand tastes as they evolve, not to dictate them. “The limit is your current taste and what you’re interested in, content will evolve alongside your taste. We see tastes shift over time; 5 years ago we saw a lot of filters and lenses flare but now much less. Tastes change, it evolves with you, you need sliding window over time, where it gets to the point of telling you what you’re going to like before you like it – we don’t see ourselves doing that.” He also pointed out how pure AI curation and self-selection bias was already causing trouble on other platforms. “You need some randomness and diversity. The Facebook echo chamber has exactly that problem because it’s something you can control; you unfriend people you don’t like.”
Yet amidst the anxiety, there was also great aspiration for what AI could do by democratizing content curation and allowing great untapped talent to be discovered. Josh related this potential to the story of Vivian Maier. “She was a nanny for 40 years in Chicago, lots of her film roles were never processed or shown to anyone. A few years back, her photos were posted online and they just sat there undiscovered.” The story didn’t end there, Josh continued “But then they were posted again in the right way with the right headline and they went viral and just last year her work had a show here in Dumbo.” Kamelia went a step further, highlighting the unprecedented accessibility of AI for mass adoption. “There are two ways to learn AI” She explained, “top down and bottom up.” Referencing the old way “bottom up you can learn basic computer science theory and algorithms and how they work and are built.” But Kamelia sees a better option “The other is top down. You can take an online course and play with libraries and open source code and actually make something.” Stressing the huge potential for AI in all fields, Kamelia concluded “This is the first time in history machine learning is so easy.”
Image Credit: CC by Gwydion M Williams
