Research and Development

1

LLM Safety as a Quality Gate: Integrating Bias, Fairness, and Robustness Evaluation into CI/CD via GitHub Apps

A GitHub App that enforces LLM safety standards at the PR level the same way linters enforce code quality.

LLM Eval Agent is the first open-source GitHub App that blocks unsafe models from merging — automatically evaluating bias, fairness, and adversarial robustness on every pull request, with confidence scoring and a full audit trail. Unlike standalone frameworks like EleutherAI Harness, DeepEval, or AIF360, it lives inside the developer workflow — no separate pipeline, no infrastructure, no manual triggers.

2

ATI: Adaptive Test Intelligence — A Unified Modular Microservice for Flaky Test Detection, Cost-Bounded LLM Repair, and Configurable Autonomy in CI/CD

ATI (Adaptive Test Intelligence) is the first system to unify flaky test detection, cost-bounded LLM repair, and configurable autonomy under a single modular REST microservice.

A live CI/CD harness across eight scenarios and seven error types confirms 100% repair rate at ≈19 s machine MTTR in Autonomous mode, eliminating unbounded human approval latency without degrading repair quality. All modules share a single SQLite persistence layer and are released as open-source under the Apache 2.0 licence.

3

CASSET: Carbon-Aware Scheduling System for Eco-Tier Testing — Risk-Proportionate Agent Tier Routing and Grid-Intensity Deferral for Agentic CI/CD

CASSET is the first framework to jointly optimise defect detection and carbon footprint in agentic CI/CD — combining risk-proportionate tier routing with real-time grid-intensity deferral under a zero-modification adapter.

World's first carbon-aware scheduler for agentic CI/CD — 38.7% carbon reduction, zero quality loss.

A five-phase evaluation across 180 real pull requests and 197 tests confirms 38.7% CI carbon reduction via tier routing and 52.4% grid intensity reduction via deferral, with zero defect detection impact (F1 = 0.931). The scheduler queries the Green Software Foundation Carbon Aware SDK to defer HIGH-tier agents to the lowest-carbon 30-minute window within a 6-hour horizon, with a hard fallback preventing unbounded latency. All components are released open-source under the Apache 2.0 licence.

4

GAMIST: Gherkin-Driven Adaptive Model-Based Intelligent Software Testing — Automating Test Model Generation from BDD Scenarios with AI-Assisted Coverage Improvement

An AI-powered quality engineering platform that transforms natural language requirements into executable Gherkin specifications, model-based workflows, and adaptive automated tests.

GAMIST is the first open-architecture framework to automate the full BDD-to-MBT pipeline — from Gherkin scenario parsing to GraphWalker JSON model generation — with LLM-driven coverage gap detection and risk-proportionate test path recommendations, deployable across any domain without model re-authoring.

The base framework combines 9 framework modules, 3 domain modules, a PAP synthesizer producing 20 vertices and 25 edges from 37 scenarios, MbtCoverageAdvisor, and CoverageGapLoop to coordinate cross-domain test coverage with measurable synthesis accuracy and production-readiness signals.

5

TOPSHAPE: Test Orchestration Platform for Service-oriented, Heterogeneous, Adaptive, Polyglot Enterprise Systems — Dynamic Test Orchestration for Real-World Business Workflows at Scale

K11 Software Solutions' Java service-orchestration framework — a reflection-driven dispatch architecture that resolves every test step to its driver class at runtime, unifying Web, Mobile, SAP GUI/Oracle EBS, API, database, and batch execution under one dynamic dispatch engine.

TOPSHAPE is a reference architecture for unifying legacy desktop/ERP automation with modern Web, API, database, batch, and messaging test layers under one dynamic dispatch mechanism and one shared execution context — driven by a single externally authored scenario file, resolved end-to-end with zero per-technology orchestration code.

The reference implementation combines a reflective ServiceResolver, a shared TestContext, a recursive ParamResolver, six technology-specific driver bases, and three client/driver factories to take a scenario from authoring to execution with convention-plus-override dispatch and cross-step data flow.

6

PRIST: Pipeline Risk-Aware Intelligent Software Testing — Autonomous CI/CD Quality Assurance Using LangGraph Multi-Agent Orchestration and Risk-Proportionate Human-in-the-Loop Control

K11tech Agentic AI QA System, a LangGraph-orchestrated multi-agent framework that autonomously executes a complete CI/CD quality gate — from pull request analysis to defect filing — without requiring manual QA intervention for routine changes.

PRIST is the first production-deployable CI/CD quality gate to combine risk-proportionate multi-agent testing, stateful human oversight, and LLM self-evaluation in a single automated pipeline — triggered by a pull request, resolved without human intervention for routine changes.

The base system combines 14 agents, 7 MCP servers, parallel dispatch, a human-in-the-loop gate, DeepEval, and RAGAS to coordinate software delivery checks with measurable risk control and production-readiness signals.

This research work extends the K11tech Agentic AI QA System — a LangGraph-orchestrated 14-agent pipeline with three interlocking innovations that close a detect-fix-learn feedback loop and transform the static quality gate into a continuously self-improving system.

Consensus Risk Scoring, Automated Remediation, and Adaptive HITL Threshold Learning

This work extends Paper 5 by introducing adaptive decision controls that quantify uncertainty, learn from reviewer outcomes, and close the loop through autonomous fix generation.

K11tech Microservice QA System, a LangGraph-orchestrated agentic pipeline that extends single-repository quality assurance to distributed microservice systems by maintaining a cross-repository view of API contracts and service dependencies.

Extends the knowledge store to capture inter-service API contracts, enabling downstream impact analysis from a single PR trigger.

This research work extends the K11tech Agentic AI QA System and K11tech Microservice QA System with six uncertainty-aware features (F1–F6) that propagate per-agent confidence through the pipeline and expose it to human reviewers in a principled way.

Continuous integration pipelines that rely on LLM-based quality agents produce binary pass/fail verdicts that discard the probabilistic uncertainty inherent in model inference. This research extends the Agentic AI QA System with six uncertainty-aware features.

This research introduces Uncertainty Source Classification: a two-category taxonomy (DATA_UNCERTAINTY, SCOPE_UNCERTAINTY) and a secondary LLM classification prompt that identifies which applies for each flagged consumer verdict in the K11tech Agentic AI QA System and K11tech Microservice QA System. Classification runs concurrently at O(n) LLM calls, adds zero pipeline latency, and never blocks the HITL gate. The two categories exhibit measurably different calibration characteristics, enabling type-stratified conformal prediction thresholds that reduce unnecessary HITL escalation.

Agentic CI/CD pipelines that produce confidence-gated verdicts tell reviewers that an agent is uncertain — but not why. Two structurally different causes of uncertainty produce identical low confidence scores yet demand different reviewer actions: DATA_UNCERTAINTY (aleatoric — evidence is genuinely ambiguous) calls for closer diff review, while SCOPE_UNCERTAINTY (epistemic — the agent lacks domain knowledge) calls for escalation to a domain specialist.