Microsoft’s AI “XiaoIce” Can Convert Images Into Beautiful Chinese Poetry

Artificial Intelligence may soon embrace the meaning of this expression “A picture is worth a thousand words”. Computers have got smarter and better in recognizing objects and are soon going to explain to us the image they saw, like what the image is about and what feeling does it reflecting.

Scientists at Microsoft Research has created a chatbot named XiaoIce. XiaoIce is a system that can automatically describe a series of images in the same way that a normal person does when he saw an image. XiaoIce knows how to interpret pictures as poems, it can write Chinese language poems inspired by images.



Margaret Mitchell, a computer scientist at Microsoft Research said: “the goal is to give AIs more human-like intelligence, to help it understand things on a more abstract level — what it means to be fun or creepy or weird or interesting.”

“People have passed down stories for eons, using them to convey our morals and strategies and wisdom. With our focus on storytelling, we hope to help AIs understand human concepts in a way that is very safe and beneficial for mankind, rather than teaching it how to beat mankind.”




XiaoIce is one of it’s kind AI, it’s the first we’ve seen that can generate Chinese language poems inspired by images:

Wings hold rocks and water lightly

in the loneliness

Stroll the empty

The land becomes soft

The face is the very first thing that we notice whenever we have to identify someone. Our brain is the world’s most powerful computer, and it has those special abilities called storing the memories in the pattern.

When you look at certain object let say a pot, all the detail that you are viewing get store in your brain in the form of patter, so when someone asks you to describe a pot, you used this patter unconsciously to describe it.




Same things happens when you look at an image and same activity get to perform when someone asks you to describe the image. Implementing the same ability in computing or electronic devices is obviously be more complex and building it into something which can also write poetry and that too in Chinese is a super complex task.

In order to build such system Scientists at Microsoft Research take the help of computer vision and deep learning artificial neural networks. To create an autonomous image description AI, Scientists has train XiaoIce to learn by example — for instance, learning how to identify cats in photos by analyzing thousands of examples of cat images.

The researchers hire workers to write sentences describing scenes consisting of five or more photos, the workers described more than 65,000 photos for the computer system. Scientists then teach the system to learn from accounts of scenes. The scientists fed system over more than 8,100 new images to examine what stories it generated.

XiaoIce is a similar system that is used for automated language translation, but instead of teaching the system to translate from one language to another, the scientists trained it to translate images into sentences

Now the question arises who will judge the poem written by XiaoIce is beautiful or not. The best and most reliable way to evaluate its quality is human judgment. Scientists have made a neural network split itself into a side that generates a poem, and a side that judges a poem.

If the judge side thinks the rules have been met and the particular poem is good enough for human eyes, it lets it through. A human then checks the results. If it’s not good enough, they go back to tweaking the parameters until it spits out the good stuff.

The computer-generated thousand of stories that would take people a lot of time and effort to examine.

When it comes to Chinese poetry, the rules have actually changed over the years. Chinese poetry involves a different set of rules and parameters, even for humans.  Microsoft researchers said working on the Chinese language bot, to write modern poetry was a pretty hard task but full of fun also.

If you want to read the paper describing XiaoIce, it is available online.

Read also, you may like it :

Scientists Trained AI To Write Poetry, It’s Standing Toe-To-Toe With Shakespeare

Scientists Train An AI To Digitally Add BIKINIS Onto Nude Photos Of Women

With AI Fox Film Studio Predicts Movie’s Audience By Analyzing Its Trailer

Artificial Intelligence (AI) Could Create 7.2 Million Jobs, More Than It Destroys – PwC Report



Leave a Reply

Your email address will not be published.