At DecisionRules, we’ve always been a bit "old-school". We built our foundation on democratizing enterprise-grade technology, and we still maintain a stubborn focus on the fundamentals. Before we build, we ask: Why? What is the reason? What is the savings? Is there any measurable benefit for the client?
Last year, we applied this pragmatic lens to the AI hype and introduced DecisionRules AI Assistant - a helper designed to guide users through the inherent complexities of modern Decision Management Platforms. We built our Assistant to solve the rule-authoring bottleneck.
The following lines detail our "build and measure" approach, outlining the methodology and the real, measurable outcomes we’ve achieved in improving the user experience after introducing AI Assistant.
The Experiment: Manual vs. Assisted Rule Authoring
For the experiment, we provided specifications / requirements for two typical decision tables (rules):
- Decision Table 1: Medium logical complexity containing between 15 to 20 conditions (rows of data)
- Decision Table 2: Medium complexity calculation where several possible approaches can be used for solving.
Our objective was to quantify the time required to author two new rules from scratch - first manually and then using the Assistant - and to compare the results. The experiment followed the performance of three test groups with different levels of expertize:
- Newbie: New user without prior experience in the tool
- Professional: Users with a working knowledge of the tool
- Expert: Experienced users
DecisionRules AI Assistant: The Results
| Experience Level | Manual Rule Authoring | 1. Elaboration of Rule by AIA | 2. Review of AIA solution | Rule Authoring with AIA | Difference |
|---|---|---|---|---|---|
| Expert | 90 mins | 20 mins | 10 mins | 30 mins | -66.7% |
| Professional | 120 mins | 30 mins | 15 mins | 45 mins | -62.5% |
| Newbie | 240 mins | 60 mins | 60 mins | 120 mins | -50.0% |
Using the Assistant reduced rule authoring time by approx. 60%, representing on a daily basis a 3x increase in productivity. This shift, however, revealed a new phenomenon that needs to be accounted for.
Comprehension Gap
The term "vibe modeling" borrows from the now-popular concept of "vibe coding" - where developers describe what they want in natural language and let AI generate the code. We applied the same principle to decision logic: users describe the business requirement conversationally, and the Assistant translates it into a working rule structure. It's fast, intuitive - but comes with a trade-off worth understanding.
While the speed gains is evident, our findings revealed a significant cognitive shift we call the comprehension gap.
When users employ AI to "vibe" their solutions into existence, they inevitably encounter a lack of inner design understanding. This is not a flaw in the AI powered design, but a natural byproduct of such automation:
- The Assistant "thinks through" the complex connections and logic branches automatically
- Whereas during the manual process, a human learns the "why" by failing, iterating, and connecting the dots. When AI autocompletes the structure, that manual "mental mapping" phase is skipped.
While the time savings were impressive, what about the quality? Not only was the AI Assistant able to create tables with more complex functions, it also identified and extracted pre-existing excel formulas to recreate in a decision table using basic operators across multiple rows of the table. From the Quality Assurance perspective, this is considered ‘Best Practices’ for decision tables and generates a table that is easier to manage, modify and understand.
More Complex Decisioning Logic
The AI Assistant is highly-capable when creating up to medium-sized rules of medium complexity, but it so far struggles to create large tables with multiple conditions (rows). That said, it can competently create the underlying logic structures so that the user can then input data using either the UI or excel import/export functionality. Where complex logic consists of multiple inbuilt functions, it is recommended to take advantage of the Assistant’s Function Expression feature which can formulate very complex functions for insertion into specific cells.
| Experience Level | Manual Rule Authoring | Rule Authoring with AIA | Difference | Cost/Hour | Hourly Productivity Gain (€) | Rule Authored per MD - Manual | Rules Authored per MD - AIA | Relative Productivity (%) |
|---|---|---|---|---|---|---|---|---|
| Expert | 90 mins | 30 mins | -66.7% | € 120/ hour | € 240/ hour | 5 | 16 | 300% |
| Professional | 120 mins | 45 mins | -62.5% | € 60/ hour | € 100/ hour | 4 | 11 | 267% |
| Newbie | 240 mins | 120 mins | -50.0% | € 30/ hour | € 30/ hour | 2 | 4 | 200% |
Conclusion
By treating AI as a practical tool rather than a mystical force, we have successfully shortened the gap between a business requirement inception and a live, functional decision. Our Assistant does not replace the human expert, it removes the manual drudgery, allowing you to focus on the Why while it handles the How.
Stay tuned, the next generation of our AI Assistant is on its way
Ondrej Brejla
Business Analyst