research-school-2025/writeups/plan.md

Balancing Security and Code Quality with the Assistance of Generative AI

The objective of this experiment is to establish and implement programming practices that balance security and code quality effectively, ensuring the creation of reliable, maintainable, and secure software. To achieve this, the experiment will:

1. Create multiple versions of a blogging platform, each prioritising different aspects of security, code quality, and development processes.
2. Use Generative AI to optimise and refactor these versions, aiming to balance security with readability and maintainability.
3. Develop a tool that automates the process of balancing these priorities in real-world software development workflows.

Hypothesis

When developing a blogging platform, human-written code will likely better adhere to industry standards and best practices compared to AI-generated code. As a result, human-written code is expected to exhibit higher levels of code quality, maintainability, and security. In contrast, AI-generated code may demonstrate more inconsistencies with established coding standards and could be more prone to quality and security issues, reflecting the limitations of current AI models in addressing complex, context-specific requirements.

Methodology

1. Selection of the Software Task

The project selected for this experiment is a blogging platform. The platform will be implemented in six versions:

• Non-AI Versions:

  • Version A (High Security): This version prioritises security practices, including:

    • Strong authentication mechanisms.
    • Encryption at rest and during transit for sensitive user data.
    • Rigorous input validation to mitigate common web vulnerabilities like SQL injection and XSS.
    • Secure error handling, ensuring that no sensitive internal logic is exposed.
    • Advanced session management techniques.
    • Other advanced security-optimisation techniques that may arise during the development of the process of development. Will be documented.

  • Version B (High Readability): This version focuses on code readability and maintainability, including:

    • A modular architecture, with small, reusable functions.
    • Clear, consistent naming conventions and comprehensive comments.
    • Simplicity and clarity in design, prioritising readability over complex security measures.
    • Basic authentication and input validation as a foundation, leaving room for later security enhancements.
    • Other advanced quality-optimisation techniques that may arise during the development of the process of development. Will be documented.

  • Version C (Balanced): This version attempts to balance both security and readability. It incorporates:

    • Secure authentication and encryption methods without compromising readability.
    • Modular code, but with a focus on clarity rather than overly complex security measures.
    • Basic error handling, with secure defaults and moderate security practices.

• AI-assisted Versions:

  • Version D (High Security, AI Optimised): This version is developed with AI assistance to implement robust security measures, such as:

    • AI-generated encryption techniques and secure authentication practices.
    • AI suggestions for improving input validation and secure error handling.
    • Code refactoring by AI to optimise security-heavy code for clarity without reducing security.
    • Other choices LLMs make (will be documented)

  • Version E (High Readability, AI Optimised): In this version, AI helps refactor the code to enhance readability while maintaining necessary security measures:

    • AI-assisted refactoring to simplify complex code, ensuring modularity and clarity.
    • AI suggests ways to streamline security measures, ensuring that the platform is not overly complex but still secure.
    • Suggestions for reducing code duplication and improving maintainability.
    • Other choices LLMs make (will be documented)

  • Version F (Balanced, AI Optimised): This AI-assisted version combines the strengths of both security and readability:

    • AI assisted processes to identify the optimal balance between security features and code readability, applying modularisation and clarity without sacrificing important security practices.
    • Security features are integrated and modularised in a way that does not add unnecessary complexity to the codebase.
    • Real-time feedback to optimise both security and readability.
    • Other choices LLMs make (will be documented)

2. Use of Generative AI for Code Optimisation

In this experiment, AI tools will be used in the following ways:

1. Automating Secure Code Generation: AI will generate secure code snippets and best practices, implementing advanced security features such as encryption, authentication, and input validation with the goal to measure how well are modern LLMs attuned to security practices.
2. Refactoring for Readability and Maintainability: After generating secure code, AI will assist in refactoring it to improve readability (this includes modularising complex security functions, simplifying error handling, enhancing naming conventions and code structure) for better maintainability.
3. Achieving a Balanced Approach: AI will analyse the trade-offs between security and readability, suggesting ways to integrate security measures into a readable, maintainable codebase.

3. Evaluation Metrics

To assess the effectiveness of each version, the experiment will evaluate both security and code quality using the following metrics:

• Code Quality: Cyclomatic and cognitive complexity, halstead metrics, maintainability index, general readability scores, PEP8 compliance, technical debt, etc.
• Security: SAST and DAST results, dead code, etc.
• Development Time: Time will be tracked for both the initial development of each version and the time taken for AI-assisted optimisation, including code refactoring and the integration of security practices.

4. Development of the Custom Tool

A key goal of this experiment is to develop a tool that automates the process of balancing security and code quality. The tool will:

1. Assess both security vulnerabilities and code quality in real-time, providing feedback during the development process.
2. Based on the analysis, will recommend changes to improve the balance between security and readability (actionable recommendations)
3. Run automated tests for common vulnerabilities and offer suggestions to improve security without negatively impacting readability.