Does it matter if companies use GenAI?

According to McKinsey, companies can expect cost decreases and revenue jumps as returns on investment in GenAI. But to paint a much clearer picture, Solita decided to test it out itself.

Along with ISS, a facility management company, our partner launched a “GenAI Twin Project” with the goal of testing GenAI capabilities in a real-life scenario of a business setting. Two teams, one armed with GenAI tools and the other serving as the control group, were tasked with developing the same digital product for the same company.

The results undoubtedly spoke in favor of the GenAI team, resulting in faster iterations, an accelerated decision-making process, fewer meetings, and less time needed to learn new skills.

GenAI challenges in software development

To make the most of GenAI for process optimization and automation, it’s important to acknowledge the limitations and challenges of this fledgling technology.

1. GenAI tools draw from already existing resources, which can impact the accuracy and security of the generated code. This limitation can cause some tools or models to miss current trends in the industry or fail to provide up-to-date solutions. Moreover, the more niche the topic, the fewer the correct answers, which is proven by the discrepancy between ChatGPT’s coding skills in web versus mobile development.
2. GenAI tools are based on probabilities, not certainties. They don’t always provide correct answers, but rather the most probable ones, which can lead to inaccurate or misleading solutions. This is especially problematic when GenAI tools are used for coding, where even basic programming problems can be misinterpreted or solved incorrectly. That risk can be limited by using smaller tools designed for specialized tasks.
3. GenAI tools hallucinate when they don’t have an exact answer. When a GenAI tool doesn’t know something, it may generate a confident yet false response. That’s why exercising caution can’t be overstated enough in software development, where relying on incorrect code or solutions can lead to critical errors. This makes the verification of generated answers of utmost importance.
4. GenAI tools require smart prompt engineering and control. To get the best answers quality-wise, it’s important to effectively communicate intentions and requirements. Breaking down tasks into smaller parts can also yield better results, along with avoiding trying to fix “bad” output with additional prompts – it’s actually better to focus on improving the original instructions.
5. GenAI tools can make managing context and validating results a challenge. To keep things organized across different parts of the project, developers have to ensure the AI correctly understands the code’s structure through documentation generation, while maintaining architectural consistency along the way.
6. Gen AI tools leverage user interactions as a training data. This means that any provided information may be incorporated into the tool’s database and potentially used in future outputs, which raises a security concern. Because of that, developers need to be aware of what information can be shared with these tools to avoid privacy breaches.

The last point is by far the most important one for businesses, since it’s easy to accidentally share information that breaches client confidentiality. Therefore, any GenAI tool should be checked for functionalities that ensure privacy. In the case of ChatGPT, that would require investing in Enterprise mode, which promises not to use the data to train the GenAI model. Another popular tool, Copilot, requires changing settings, which are turned on by default to use your data.

Future Mind & Solita Group’s countermeasures

Aware of these challenges, at the Solita Group, including Future Mind, we have introduced countermeasures to ensure the highest quality of our digital solutions:

1. Raise awareness of GenAI risks and educate on best practices by providing access to trainings and lectures, as well as encouraging sharing knowledge within the teams.
2. Only use tools that are approved for business use and ensure confidentiality, such as ChatGPT Enterprise, GitHub Copilot, M365 Copilot, Azure Open AI.
3. Only use tools that are compliant with EU AI Act readiness and fall into the category of “minimal risk” GenAI systems.
4. Respect clients’ policies and restrictions on GenAI; no GenAI use is allowed if a client forbids it, no matter the reason.
5. Only use tools approved by clients for each project for full transparency.
6. Each GenAI system has an assigned owner to ensure accountability and responsibility, and also to avoid confusion and conflicts between the teams.
7. Carefully monitor the GenAI output for mistakes and discriminatory bias, and don’t allow GenAI to make any decisions without a human expert’s oversight.

Where GenAI shines the brightest: most common uses

GenAI tools are far from perfect. There are still some areas that leave much to be desired, while in others, they outperform specialists. Based on a survey conducted among the Future Mind & Solita Group employees, we’ve pinpointed several use cases where GenAI turns out to be most useful.

• Brainstorming

GenAI tools can generate a huge amount of content in just a few minutes, making it great for coming up with ideas and good starting points. After all, for many creatives – including developers – there’s nothing worse than looking at an empty file. With the help of GenAI, you can quickly jumpstart your project by skipping ahead in the conceptual phase.

You can also use GenAI tools to clash different ideas together. While GenAI hardly ever comes up with truly brilliant ideas, it can give you an outsider perspective on your problems and solutions. It can also point out the basic issues that can be easily forgotten while thinking of the bigger picture.

• Quick replacements

Coding includes manual tasks that are necessary, yet time-consuming. For example, if one value needs to be changed in almost finished code, and it appears throughout the whole project, a developer may have to replace all these values manually if they lack the right tools. With GenAI, you can ask it to replace all these values for you. And when it knows the context of the entire project, it can make fitting adjustments to take the rest of the code into consideration.

• Recycling

There’s really no reason to reinvent the wheel with every new project. Reusing older pieces of code – as long as they don’t fall under any copyright – can accelerate the development process by leaps and bounds. Moreover, it decreases the chance of having bugs or any inconsistencies, since the code was already tested.

GenAI can only make reusing the code even easier with real-time recommendations, streamlined updates, dependency insights, and automation.

• Code review

Code review, while undoubtedly a necessary part of any development, can stall any further progress awaiting the feedback – team members stuck in meetings or simply enjoying their holidays can throw a wrench into any plans.

To optimize the process, an increasing number of helpful tools are emerging to help analyze code, provide feedback, remind developers of best practices and common pitfalls, as well as ensure consistent quality across the entire project. Of course, the last word will always belong to a fellow specialist, but GenAI tools can facilitate the process and catch the most obvious mistakes.

• Generating testing scenarios

In quality assurance, the answers to “what the user won’t do” are much fewer than to “what the users will do”. In the case of the latter, the possibilities might be nearly endless, and it’s a given that unexpected edge cases will appear.

Thankfully, GenAI tools can be used to generate hundreds of detailed scenarios based on specified requirements and user stories, making sure no cases are left untested.

• Onboarding

Introducing new team members to an already existing project is always a challenge, especially when new languages are involved. The whole team has to ensure the newcomer is up-to-speed and understands the decisions that influence the code. Without it, the newly written code may be inconsistent with the rest of it.

GenAI can help out in this case as well. When prompted, it can give exhaustive explanations about the code, which can be adjusted to fit any circumstances and the level of knowledge. It can also remind us of the practices we used based on the context of the whole project.

Creativity is humanity’s strongest suit

Is there anything the GenAI tools can’t do?

Absolutely. For example, because GenAI is based on probability, it won’t come up with truly unique ideas, or such that could be considered interdisciplinary. After all, GenAI simply regurgitates what it was already told. But it can be used in reverse, to see what kind of ideas are most common or to evaluate our own.

Moreover, the more unusual the project, the more difficult it will be for GenAI tools to follow. It also applies to the newest technologies; since they aren’t as well documented as the older ones, the GenAI won’t have the required resources to generate proper answers from.

Conclusions

GenAI tools have become commonplace amid the Future Mind & Solita Group employees, with ChatGPT and Copilot being the most often used ones. Still, because the technology lacks maturity, it requires further experimentation and testing to become a truly dependable part of any development process. But even the way it is now, GenAI already helps us achieve impressive results, from previously unseen productivity and efficacy levels to reducing time spent on redundant tasks.

But most importantly, these achievements translate into faster development and higher quality of developed solutions. By leaving the repetitive and time-consuming tasks to GenAI, developers can focus on the crux of the project and spend more time coming up with creative solutions. This also means that more time can be devoted to quality assurance, double and triple checking more scenarios, ensuring that no stones were left unturned.