Researchers from Intel Labs in Israel developed the NeuroPrompts system in which the user enters a simple prompt, such as "a boy on a horse". The system takes this basic prompt and automatically upgrades it to a more complex prompt that produces better quality images. The process is based on a combination of supervised learning and reinforcement learning
Since the launch of ChatGPT in the fall of 2022, prompt engineering has become a major topic in the world of artificial intelligence. Many people have tried to find the best way to formulate queries for large language models (LLMs) or for image and video generation using AI. The internet is full of guides, reference pages and advice for improving the use of these models.
However, new research suggests that prompt engineering may be more effective when performed by the model itself, rather than by a human engineer. These findings raise doubts about the future of the prompt engineering profession, at least in its current form.
In the last decade we have experienced an impressive technological revolution in the field of artificial intelligence, which has fundamentally changed the face of many fields. Breakthroughs in large models, such as GPT-3 and GPT-4, resulted in a significant improvement in computing capabilities, natural language processing and image recognition. However, the large models also have significant limitations, including high costs and complex understanding and control.
Against this background, a new research direction is developing that focuses on the automation of the prompt engineering process itself. This approach promises not only to improve the quality of the prompts, but also to make the technology more accessible to a wide range of users, beyond AI experts. This is where a system comes into play NeuroPrompts, a development from Intel Labs, which offers an advanced solution for automated prompt engineering.
A team of researchers led by Shahar Rosenman, Vasudev Lal and Philip Howard developed the system NeuroPrompts. In a previous interview, Vasudev Lal, Principal Research Scientist at Intel Labs, explained that the goal was to develop a system that could bridge the gap between users' creative ideas and the advanced capabilities of Modly.AI. They wanted to make this technology more accessible to anyone, regardless of background. His technical
NeuroPrompts works as follows: the user enters a simple prompt, such as "a boy on a horse". The system takes this basic prompt and automatically upgrades it to a more complex prompt that produces better quality images. The process is based on a combination of supervised learning and reinforcement learning.
Shahar Rosenman Explains the development process. "First, we trained a language model on a huge pool of expert-generated prompts. Then, we used a PPO (Proximal Policy Optimization) algorithm to further improve the model based on estimated human preferences."
One of the impressive innovations in NeuroPrompts is the use of a technique called NeuroLogic Decoding. This technique allows the system to generate improved prompts while maintaining certain constraints, such as a specific artistic style or a certain atmosphere. This gives Intel researchers the ability to control the creation process more precisely, allowing users to get results that better match their vision, without losing the benefits of automation.
In experiments conducted, images created from NeuroPrompts-enhanced prompts achieved an average aesthetic score of 6.27 out of 10, compared to 5.64 for raw prompts and 5.92 for prompts created by human experts. Furthermore, the system achieved a 20% improvement in PickScore, which predicts human preference for the generated images.
The NeuroPrompts interface includes a side-by-side comparison of the images created from the original and the improved prompt, accompanied by aesthetic scores and PickScore. This allows users to easily see the difference and understand the improvement that the system brings.
practical applications of NeuroPrompts
Intel Labs' NeuroPrompts system is already showing its capabilities in a variety of fields, from art to medicine. In a pilot project in collaboration with a leading art gallery, they used NeuroPrompts to create an entire exhibition of digital art. Visitors were invited to enter simple prompts based on personal feelings and memories, and the system transformed them into complex and moving digital artworks.
In the field of education, an experiment was conducted to teach history, where teachers used the system to create more accurate historical images. For example, the prompt "French Revolution" was expanded to a detailed description of a mob storming the Bastille, including accurate historical details.
In the film industry, directors will be able to use NeuroPrompts to plan complex scenes. Simple prompts expanded into detailed descriptions will allow production teams to visualize and plan scenes efficiently.
In the medical field, NeuroPrompts is in early testing stages to aid in the interpretation of medical images, helping radiologists expand simple descriptions of findings into more detailed descriptions.
Additional studies in the field
Intel Labs is not alone in the race to develop automated prompt engineering technologies. Researchers from VMware, led by Rick Battle and Teja Golfudi, conducted a comprehensive study on the effect of different strategies in prompt engineering. They found that prompts based on popular culture references, such as Star Trek references, can improve performance on math tasks. VMware's research provides interesting insights and shows that there is much more to explore in this area. While Intel Labs focuses on imaging, the principles can be applied to a wide variety of fields.
Ethical challenges and considerations
One of the main challenges in developing systems like NeuroPrompts is ensuring fairness and ethics. Rosenman emphasizes that they are aware that AI systems can reproduce and amplify social biases. They work hard to minimize this risk, for example by diversifying their data sources and rigorously vetting the results.
The increasing use of technologies like NeuroPrompts also raises additional complex ethical questions. For example, there are concerns about the impact of these systems on employment, privacy, and copyright. Ethics experts emphasize the need to develop regulatory frameworks that will ensure responsible use of these technologies.
Despite the obvious advantages, there are also criticisms of the automated approach to prompt engineering. Critics argue that over-reliance on automated systems could impair human creativity and lead to excessive standardization of AI products.
In response to these criticisms, Lal emphasizes that they are not trying to replace human creativity. The goal is to give better tools to creative people. NeuroPrompts, he says, is like a sophisticated digital assistant that helps get the best out of technology.
As technology advances, it seems that the future of prompt engineering and creation using AI is likely to be both fascinating and challenging. The balance between automation and human creativity will undoubtedly be one of the main issues that will occupy the technology industry in the coming years.
What is certain is that Intel Labs and other companies in the field will continue to research and develop technologies that will further improve the capabilities of automated prompt engineering.
In conclusion, the automatic prompting revolution, as exemplified by Intel Labs' NeuroPrompts, marks an exciting new era in the development of artificial intelligence. It promises not only to improve the quality of the products of AI systems, but also to make the technology more accessible to a wide range of users. As Shahar Rosenman points out, "The future of working with artificial intelligence is not just about letting computers do everything on their own. Instead, the idea is to connect what people are good at - such as creativity and new ideas - and what computers are good at, such as fast information processing. When combined These two things together, you can create new things, think of original solutions, and solve big problems that affect the whole world. It's an opportunity to do things that we couldn't do before."
More of the topic in Hayadan:
