Back To Projects

Generating Instagram Captions with ViT-GPT2 and GPT3

Ariel M.

This paper presents a novel approach for generating Instagram captions based on visual features and language models. Our caption generator combines Vision Transformer-GPT2 and GPT3 to generate descriptive and engaging captions in the style of an Instagram post.


This paper presents a novel approach for generating Instagram captions based on visual features and language models. Our caption generator combines Vision Transformer-GPT2 and GPT3 to generate descriptive and engaging captions in the style of an Instagram post. Our caption generator has the potential to assist social media users, content creators, and marketers in generating high-quality and engaging captions for their Instagram posts. Future work includes exploring more advanced visual and textual features and incorporating user feedback and preferences. Instagram is a popular social media platform for sharing photos and videos. One of the key elements of a successful Instagram post is a catchy caption that enhances the visual content and engages the audience. However, coming up with a good caption can be challenging, especially for users who are not native speakers of the language or lack creativity. In this paper, we propose an automatic caption generator that takes an image as input and generates an Instagram-style caption using state-of-the-art language models.

Ariel M.
Roger Jin
Computer Science MS at MIT, ML researcher at Apple

Related Projects

Predicting Future Phonological Changes Of Mandarin Chinese

We created a system of translating IPA representation to vectors that captures characteristics of phonemes such as their articulatory location, and experimented with several machine learning models to capture existing trends from Old, Middle to Modern Chinese (Mandarin).
Peijie G.
Mentored by Kush Khosla
Generation of Research Paper Titles

Can NLP accurately and effectively generate research paper titles? In this research paper, an effective and accurate artificial intelligence NLP model is tried to be determined by evaluating various models and methods for title generation.
Christopher G.
Mentored by Sean Konz
Building an Optimized algorithm that provides summaries of legal documents

The legal industry is built around documents as they provide evidence and reduce doubt in the court. Due to the large volume of documentation in the legal industry, the processing and summarization of these documents is important to a number of individuals. We were able to create a user interface that allows for the input of documents and makes use of the algorithm we created to output a summary of the document which can be copied by the user.
Aman B.
Mentored by Eric Bradford