News

Microsoft’s ‘CaptionBot’ Adds Incorrect Captions to Your Favourite Pictures

Microsoft, as part of its new research into storytelling by artificial intelligence, has released CaptionBot, an AI designed to recognise images and add an appropriate descriptive caption. However, like its previous attempt at AI – chatbot Tay – CaptionBot isn’t entirely successful. As with Tay, though, the results are hilarious (and without any fascistic or incestuous overtones).

The accompanying academic paper, titled Visual Storytelling [PDF], describes how the Microsoft Sequential Image Narrative Dataset (SIND) applies value judgements to picture content, setting, composition, and human expression in an attempt to describe the scene. The paper adds:

“There is a significant difference, yet unexplored, between remarking that a visual scene shows “sitting in a room” – typical of most image captioning work – and that the same visual scene shows “bonding”. The latter description is grounded in the visual signal, yet it brings to bear information about social relations and emotions that can be additionally inferred in context.”

To set CaptionBot’s base level, 10,117 CC-licensed Flickr albums were ploughed through by Amazon Mechanical Turks, who assigned tradition captions to a series of pictures. An ‘average’ description of each picture was derived by the multitude of entries, and that average was reduced to an algorithm that CaptionBot could apply to fresh images in order to evaluate them.

“Captioning is about taking concrete objects and putting them together in a literal description,” Margaret Mitchell, lead researcher on the project, said in a Microsoft blog post. “What I’ve been calling visual storytelling is about inferring conceptual and abstract ideas from those concrete objects.”

Ashley Allen

Disqus Comments Loading...

Recent Posts

Cooler Master HAF 700 Full Tower PC Gaming Case inc 5 ARGB Fans

To unlock their true potential, cooling components must work, not individually, but together as an…

3 mins ago

MSI Intel PRO B760-P WIFI DDR4 ATX Motherboard

Meet the PRO B760-P WIFI DDR4, here to support the very latest 13th Generation Intel…

4 mins ago

JBL Quantum 810 Wireless Gaming ANC Headset Bluetooth/RF Black PC/Console

Feel like you're stepping inside the game with JBL QuantumSURROUND and DTS Headphone:X version 2.0…

6 mins ago

Corsair Vengeance RGB Black 32GB 5600MHz DDR5 Memory Kit

CORSAIR VENGEANCE RGB DDR5 memory delivers DDR5 performance, higher frequencies, and greater capacities optimized for…

9 mins ago

Samsung 970 EVO PLUS 1TB M.2 NVMe PCIe Performance SSD/Solid State Drive

Advance into the next generation of computing with the Samsung 970 EVO Plus SSD, specially…

10 mins ago

WD Black SN850X 1TB M.2 PCIe 4.0 Gen4 x4 NVMe SSD PC/PS5

Strap in for breakneck gaming speeds with the WD_BLACK SN850X NVMe™ SSD. Crush load times…

12 mins ago