Artificial Intelligence (AI) holds tremendous promise for society, bringing forth a myriad of benefits. However, the realization of AI’s full potential hinges on its secure and responsible development, deployment, and operation. Cybersecurity emerges as a critical prerequisite, serving as the foundation for ensuring the safety, resilience, privacy, fairness, efficacy, and reliability of AI systems.

In this dynamic landscape, the development of AI systems introduces novel security vulnerabilities that demand attention alongside conventional cybersecurity threats. Amid the rapid pace of AI evolution, security considerations must not take a back seat. Instead, security becomes a non-negotiable core requirement, intricately woven into the fabric of the AI system’s entire life cycle.

At Tier3 Cybersecurity Pakistan we delve into the imperative fusion of AI and cybersecurity, ensuring a future where innovation meets responsibility head-on. Exploring this intricate dance between advancing AI capabilities and safeguarding against potential vulnerabilities. In the ever-evolving realm of technology, security isn’t just a feature; it’s the bedrock of responsible AI development.

Artificial intelligence (AI) and machine learning (ML) systems are increasingly used in every aspect of life, from providing the ‘smart’ in your smartphone, to marketing and research, through to critical areas like healthcare, finance and national security.

As its use continues to grow, as users we need to know ML is being deployed securely, without putting our personal safety or data at risk. At its foundation, software security relies on understanding how a component or system works. This allows a system owner to test for and assess vulnerabilities, which can then be mitigated or accepted.

AI systems are subject to novel security vulnerabilities that need to be considered alongside standard cyber security threats. When the pace of development is high – as is the case with AI – security can often be a secondary consideration. Security must be a core requirement, not just in the development phase, but throughout the life cycle of the system.

Unfortunately, it’s hard to do this with ML. ML is used precisely because it enables a system to learn for itself how to derive information from data, with minimal supervision from a human developer. Since a model’s internal logic relies on data, its behaviour can be difficult to interpret, and it’s often challenging (or even impossible) to fully understand why it’s doing what it’s doing.

As a result, ML components usually don’t have the same level of scrutiny as standard systems, and ML vulnerabilities can be missed.

Depending on which algorithm and architecture is used, you may still not be able to fully understand or explain a model’s logic, even if you have free rein to inspect it. Explainable AI (XAI) is a sub-field of AI that helps humans interpret the results of their models, for reasons relating to ethics and performance, as well as security. It is, however, still a developing field. To get around our inability to understand a model’s logic, perhaps we can understand its behaviour by providing various inputs, and seeing how the model reacts.

System owners and senior leaders understand threats to secure AI and their mitigations. The data scientists and developers should maintain an awareness of relevant security threats and failure modes and help risk owners to make informed decisions. Providing users with guidance on the unique security
risks facing AI systems (for example, as part of standard InfoSec training) and training developers in secure coding techniques and secure and responsible AI practices.

When choosing an AI model, your should consider

• the complexity of the model you are using, that is, the chosen architecture and number of parameters; your model’s chosen architecture and number of parameters will, among other factors, affect how much training data it requires and how robust it is to changes in input data when in use
• the appropriateness of the model for your use case and/or feasibility of adapting it to your specific need (for example by fine-tuning)
• the ability to align, interpret and explain your model’s outputs (for example for debugging, audit or regulatory compliance); there may be benefits to using simpler, more transparent models over large and complex ones which are more difficult to interpret
• characteristics of training dataset(s), including size, integrity, quality, sensitivity, age, relevance and diversity
• the value of using model hardening (such as adversarial training), regularisation and/or privacyenhancing techniques
• the provenance and supply chains of components including the model or foundation model, training data and associated tools

The production of comprehensive documentation supports transparency and accountability. Your documentation should include the creation, operation, and life cycle management of any models, datasets and meteor system-prompts. Your documentation must have security-relevant information such as the sources of training data (including fine-tuning data and human or other operational feedback), intended scope and limitations, guardrails, cryptographic hashes or signatures, retention time, suggested review frequency and potential failure modes. Useful structures to help do this include model cards, data cards and software bills of materials (SBOMs).

Attackers may be able to reconstruct the functionality of a model or the data it was trained on, by accessing a model directly (by acquiring model weights) or indirectly (by querying the model via an application or service). Attackers may also tamper with models, data or prompts during or after training,
rendering the output untrustworthy.

You can protect the model and data from direct and indirect access, respectively, by:

• implementing standard cyber security best practices
• implementing controls on the query interface to detect and prevent attempts to access, modify, and exfiltrate confidential information

To ensure that consuming systems can validate models, you compute and share cryptographic hashes and/or signatures of model files (for example, model weights) and datasets (including checkpoints) as soon as the model is trained. As always with cryptography, good key management is essential.

At Tier3 we participate in information-sharing communities, collaborating across the global ecosystem of industry, academia and governments to share best practice as appropriate. As the ‘provider’ who is responsible for data curation, algorithmic development, design, deployment and maintenance we maintain open lines of communication for feedback regarding system security, both internally and externally to our organisation, including providing consent to security researchers to research and report vulnerabilities.

When needed, users escalate issues to the wider community, for example publishing bulletins responding to vulnerability disclosures, including detailed and complete common vulnerability enumeration. We
take action to mitigate and remediate issues quickly and appropriately.