Self-assessment
Score how well your organization follows each control, add your own notes, and open the radar chart to see where you are strong or thin. Your scores and notes are saved only in this browser, so nothing is shared or sent anywhere. This is an informal, qualitative exercise with no official score; for a formal, interview-based assessment see Measure and Improve.
This browser is blocking storage, so your scores will be lost when you leave the page. Use Your data → Export to keep a copy.
Your data
Governance
Product
Environment
Deployment
G.1 Perform compliance
-
G.1.1 Organizational security requirements
Identify, document, communicate, and maintain security requirements and policies for the organization's software development infrastructure, including all AI models used, and secure SDLC. Maintain the requirements and policies over time. Incorporate constraints imposed by standards and regulations and customer-driven security requirements.
Assessment questions
- How do you document and share a secure SDLC that the engineers are aware of?
- How do you define security requirements and policies for the organization, its development infrastructure, contributions, and processes?
- How are these requirements and contributions maintained over time?
- How are constraints imposed by regulatory and compliance drivers included in these requirements, policies, and the SDLC?
-
G.1.2 Software license conflict
Software licenses may or may not allow certain types of usage, contain distribution requirements or limitations, or require specific action if the software is modified. Risk is increased if the licenses of components conflict with an organization's policies. Software licenses should be documented and tracked to trace the users and use of licenses to access control information and processes according to software usage restrictions. License metadata should be recorded during the build and made available in the SBOM.
Assessment questions
- How do you check that the software licenses for tools or third party components comply with your organization's use policies?
- What tools, if any, do you use to automate the license check process?
- How do you document and track users and uses of software licenses relative to access control policies and software usage restrictions?
-
G.1.3 Produce attestation
Configure tools to generate attestations (aka evidence) to create an audit trail for every step in the build process and of the use of secure software development practices. These attestations should indicate conformance with record retention requirements, preserve the integrity of the findings and the authenticity of the information. For AI models, generate attestations of the integrity and provenance of the training datasets. Assign responsibility for creating attestations that tools cannot generate. Attestations should be immutable and published in the source repository releases, in the package registry, or elsewhere, with their existence recorded in a transparency log.. An attestation is a signed assertion that the record is trustworthy.
Assessment questions
- Describe your automated or manual processes for producing attestations for the build process, the integrity/provenance of training data, and for the use of secure development practices?
- Where do you publish your attestations, e.g. source repositories, package registries, transparency logs?
- What are your auditability/retention requirements for these attestations?
- How is responsibility assigned for creating needed attestations that tools cannot generate?
- What tools or frameworks (e.g. TUF, in-toto) do you use to produce authenticated attestations?
- What is your rationale for providing self-attestation for your product?
- How do you ensure that the attestations are immutable?
-
G.1.4 Deliver provenance
Automatically collect and provide traceability data for generated products, including AI models, including: what entity built the package, what build process they used, and the top-level inputs to the build, such that the data's traceability can be verified and is generated automatically from the build service using a convention common in the package ecosystem. Provenance SHOULD accompany the artifact at publish time, and package ecosystems SHOULD provide a way to map a given artifact to its corresponding attestations.
Assessment questions
- Describe your manual and/or automated processes for generating provenance data.
- What authentications take place relative to the production of provenance data?
- How is provenance data verified?
- How is the integrity and immutability of provenance data managed?
- What do provenance records include, e.g. the initial state of the machine, VM, or container, and user-specified build steps?
- How do you ensure that access to generated AI models trained on sensitive data (e.g. PII, IP, financial, or other regulated data) is restricted to individuals who already have access rights to the sensitive data or that generated models do not contain sensitive data?
-
G.1.5 Deliver SBOM
Generate (preferably during build time) and provide a SBOM (in a machine-readable, NTIA-supported format) for generated products, including AI models. SBOMs should be digitally signed using a verifiable and trusted key. VEX data for all components should also be considered.
Assessment questions
- How do you deliver an SBOM and provenance data that satisfy your contracts with software acquirers, such as in a standard-base format?
- How and when is the SBOM built?
- What tool do you use?
- How do you digitally sign the SBOMs?
- How do you produce a VEX supplement for your SBOM, if any?
- What is your process for generating the VEX information, if any?
G.2 Develop security policies
-
G.2.1 Upper management support
An upper management (e.g., c-suite) leadership team understands the business risk of insecure software development, including AI models used and delivered, and of violating compliance and privacy obligations. The team is responsible for the entire software development process and deploying secure software to production. The commitment of this team is communicated to personnel associated with development-related roles and is backed up by the commitment to allocating adequate resources and making sometimes-difficult business decisions for security over faster release dates.
Assessment questions
- Please describe upper management's commitment to secure software development practices involving AI models.
- In what way is this commitment publicly demonstrated and communicated to development roles and responsibilities?
- Comment on the adequacy of resources dedicated to secure software development.
-
G.2.2 Secure SDLC checks
Define criteria for a secure SDLC and associated software security checks that indicate how effectively software, including AI models, resulting from the SDLC meets the organization's expectations. These checks include key performance indicators (KPI), vulnerability severity scores, and security checks included in the 'definition of done' in an agile process and may be used for go/no-go decisions. The use of automated tools aids thoroughness, objectivity, and efficiency of these checks.
Assessment questions
- What kind of criteria for security checks has been established, such as for security testing results?
- How can these checks indicate if security practices are being used and secure software is being developed?
- How are these checks tracked through the SDLC?
- Are the checks automated or manual?
-
G.2.3 Roles and responsibilities
Throughout the organization, create new roles and alter responsibilities for existing roles to incorporate security controls and practices, as appropriate, into the SDLC and to keep management educated and informed on security issues. These roles can be centralized for the organization to promote thought leadership among developers and architects; and distributed throughout the organization in a network of security advocates.
Assessment questions
- What security roles and responsibilities have been created to encompass all parts of the SDLC - both within the development teams as well as at an organizational level?
- What security roles and responsibilities have been created to address AI model acquisition, development, use, and publication?
-
G.2.4 Security code review policy
The policies for whether security-focused code review (a person looks directly at the code to find issues) and security-focused code analysis (tools are used to find issues in code, either in a fully automated way or in conjunction with a person) should be conducted based upon the characteristics/criticality of the software and its stage of development. Policies are established for code and AI models developed in-house and for third-party code and AI models.
Assessment questions
- What kind of guidelines are set in place for which code and models should be manually or automatically reviewed from a security perspective?
- How are these guidelines communicated and enforced?
-
G.2.5 Asset inventory
Maintain a system component inventory, including hardware, software licenses, software versions, direct and transitive component owners, containers, machine names, and network addresses. Maintain an operations software inventory, including a map of source code; open source incorporated during the build and dynamic production; software deployments and related containerization, orchestration, and deployment automation code with respective owners. Unique identifiers for the inventoried assets should be established. Suppliers should also produce an asset inventory, for example through generating SBOMs. Particular attention should be placed when a product or system is retired.
Assessment questions
- How is an asset inventory automatically or manually generated and maintained?
- How do you integrate vulnerability management with asset inventory management, for example through SBOMs or software composition analysis?
- What process is used for the asset inventory to be updated when a product or system goes to end-of-life?
-
G.2.6 Protection of information at rest
Data protection at rest aims to secure the confidentiality and integrity of data stored on any storage device or network. Mechanisms to achieve confidentiality and integrity protections include using encryption and file share scanning. Provisions for protecting information at rest should be included in agreements with suppliers, developers, system integrators, external system providers, and other service providers.
Assessment questions
- What provisions are made for the protection of information at rest included in agreements with suppliers, developers, system integrators, and external system providers?
- What policies are set in place for the use of cryptography in the organization’s security policy?
G.3 Manage suppliers
-
G.3.1 Security-related contract terms
Compliance requirements, security requirements, and secure SDLC practices are included in vendor contracts with specified means of adherence enforcement. Examples include supplying an SBOM, VEX, self-attestation of security practices and provenance information; having a vulnerability disclosure program and incident response plan; and a security training program. The vendors include component, cloud, AI model, middleware providers, container and orchestration providers, and contractors.
Assessment questions
- What requirements language is included in system and services acquisition policies and contracts related to adherence to security policies, security requirements, and secure SDLC practices compatible with compliance requirements?
- How Do you obtain SBOMs, provenance data, and attestation of adhering to security practices from your suppliers?
- How is adherence to contract terms verified?
- How do you track the provenance of training, testing, fine-tuning, and aligning data used for AI models?
-
G.3.2 Separation of duties
Ensure that appropriate separation of duties is established for decisions that require the acquisition and administration of information systems and the acquisition of components entering the supply chain. Separation of duties can be used to deny contracted developers the privilege to promote code they wrote from development to the production environment. Separation of duties can prevent collusion, for example, by ensuring personnel administering access control functions do not also administer acquisition.
Assessment questions
- How is separation of duties established for decisions that require the acquisition of information systems and supply chain components, such as components entering the enterprise's supply chain or contracted developers promoting code from development to production?
-
G.3.3 Information disclosure
Unauthorized disclosure of information is a form of data leakage. Monitoring should be in place for contractor systems to detect the unauthorized disclosure of any data, and contract language should include a requirement that vendors notify of unauthorized disclosure of information.
Assessment questions
- What kind of contract language is in place to stipulate that contractor systems conduct monitoring to detect unauthorized disclosure of any data?
- What kind of contract language is in place to require that vendors notify of unauthorized disclosure?
-
G.3.4 Session audits
Include contract employees and prime contractors in session audits to identify security risks in the supply chain. Session audits can include monitoring keystrokes, tracking websites visited, and recording information or file transfers and may involve implementing specialized session capture technology. As such, the privacy risks of session audits should be considered, and session audits may only be activated under certain circumstances, e.g., the organization is suspicious of a specific individual.
Assessment questions
- Under what circumstances are contract employees included in audits to identify security risks in the supply chain?
-
G.3.5 Notification agreement
Require suppliers to establish agreements and procedures for notification and monitoring capabilities, including notification of being the target of a supply chain threat. Timely notification of compromises and potential compromises in the supply chain is essential for an organization to initiate an incident response. Establish a minimum amount of time a vendor must declare that a product will be declared end-of-life and will no longer be supported and understand what end-of-life options exist (e.g. replace, upgrade, migrate to a new system, etc.).
Assessment questions
- What notification agreements and monitoring capabilities are established with your suppliers related to supply chain threats or incidents?
- How much notification does a vendor provide when a product goes end-of-life?
- What end-of-life options exist?
G.4 Training
-
G.4.1 Role-based training
Provide literacy and role-based training on software security and secure software supply chain. This training should include information about the firm's security culture, the secure software development lifecycle, containerization and security orchestration, common security mistakes, technology such as CI/CD, AI models, and DevSecOps, and recognizing insider threat. Require this training as part of onboarding and periodically.
Assessment questions
- Describe the training given to employees and to vendors related to software supply chain security?
- Describe your process for scheduling and monitoring team member training in secure software architecture, design, development, testing, and software supply chain management.
- How do you customize the training specific to team roles, development tools, and languages?
-
G.4.2 Contingency training
Conduct contingency training to prepare for emergency response, backup operations, and post-disaster recovery to ensure the availability of critical information resources in emergencies. Critical suppliers should also be included in the training.
Assessment questions
- Describe your contingency training for preparing for emergency response, backup operations, and post-disaster recovery.
-
G.4.3 Gather attack trends
Have a process for continuously learning about emerging attack trends and vulnerabilities. Gather information from software acquirers, users, and public sources on potential vulnerabilities in the software and third-party components that the software uses, including AI models, and investigate all credible reports. Use a subscription to a cyber threat intelligence feed, attend technical conferences, monitor attacker forums, and study trends within the enterprise. Make this information on attack trends actionable and useful for developers, testers, security operations, and others to identify vulnerabilities in existing products, perform improved threat modeling and security architecture, and evolve the SDLC.
Assessment questions
- How do you monitor new attack trends and vulnerabilities applicable to your software and AI models, such as by monitoring a cyberthreat intelligence feed and attacker forums, or attending conferences?
G.5 Assess and manage risk
-
G.5.1 Criticality analysis
Perform a criticality analysis of system components, functions, or services to assign cybersecurity supply chain risk management activities commensurate with the analysis based upon the likelihood and impact of an attack. Not all system components, functions, or services necessarily require significant protection. Items to consider in the analysis include system assets/data involved in the product, applicable laws, regulations, policies, standards, system functionality requirements, system and component interfaces, and system and component dependencies. The criticality analysis impacts the procedures contractually imposed on vendors.
Assessment questions
- What process do you have for performing criticality analysis as input to assessments of supply chain risk management activities?
-
G.5.2 Track security risks and decisions
Record when a design tradeoff, vulnerability decision/exception, or component choice has been made that incurs security risk and a mitigation plan to reduce that risk including with vendors. Periodically re-evaluate these exceptions and mitigation plans, potentially including a review board that can provide security guidance in design guidance, patterns, standards, features, and frameworks.
Assessment questions
- How are the responses to security risks and design decisions recorded, including how mitigations are to be achieved?
- How are approved exceptions to the security requirements periodically evaluated?
-
G.5.3 Security metrics
Identify and regularly review metrics to measure the effectiveness of the software security program, including establishing Key Performance Indicators (KPIs). The regular review can drive budgeting and resource allocations and measure performance against risk appetite and risk tolerance statements. Publish data/dashboards about the state of software security within the organization. Automate metrics collection via security telemetry, as possible, to gather measurements to enhance efficiency and objectivity.
Assessment questions
- What security metrics do you track to indicate how you are doing relative to developing secure software products and having an effective and efficient software security program (e.g., security outcomes, MTTR)?
- Are these metrics collected automatically or manually?
-
G.5.4 Data-informed product decisions
Implement processes and mechanisms, automated when possible, to gather and safeguard information in support of security decision-making at the product level. This information can be used to drive SDLC change at the product or organizational level and/or drive a security risk exception process.
Assessment questions
- How do you use information from your toolchain to informs security decision-making automatically?
- If not, is this information collected and reviewed manually?
P.1 Develop security requirements
-
P.1.1 Product security requirements
Identify and document security requirements for organization-developed software, including AI models, including risk-reducing software architecture and design choices, security patterns, and translating compliance constraints to requirements. Examples include using memory-safe languages and secure frameworks, isolation and sandboxing component strategies, code modularity, security features, secure-by-design components, application containers, and product features that aid in secure deployment, operation, and maintenance. Containers can be used as a strategy for tighter coupling of an application and its dependencies, immutability, and some isolation benefits. Maintain these requirements over time.
Assessment questions
- How are risk-reducing security architecture and design requirements for products considered and developed?
- Which of the following strategies are considered: the use of memory-safe languages, secure frameworks, isolation, acceptance testing, and sandboxing?
- How are completion and adherence tracked?
-
P.1.2 Software release integrity
Use code protection mechanisms, such as the use of an established certificate authority for code signing, to enable the attestation of the provenance, integrity, and authorization of important code and AI models. Make software integrity verification information, such as cryptographic hashes for release files, available to software acquirers on a well-secured website.
Assessment questions
- What software integrity artifacts do you make available to software acquirers, such as cryptographic hashes of release files?
- What certificate authority do you use for code/commit signing?
- How often do you review code signing processes, such as certificate renewal and key rotation?
- How do you manage and protect your signing server, for example is it dedicated?
P.2 Build security in
-
P.2.1 Security design review
Enumerate possible threat vectors. Conduct threat modeling and attack surface analysis to identify weaknesses in the software architecture and design such that the system is resistant to attack. Identify missing security features and requirements. Identify unused components in the design. Address the identified security risks through re-design, or develop and track a mitigation plan for exceptions.
Assessment questions
- For all critical software components and external services that your team operates and owns, has a qualified person not involved in the design reviewed the design and conducted an attack surface analysis and threat model?
- What kind of approaches are you using to narrow attack vectors?
- What kind of tool or methodology (like STRIDE) do you use to structure your threat modeling?
- Do you perform analysis such that only required modules are included in the product and unused modules are uninstalled and removed, decreasing the attack surface?
- How are unused modules identified - such as 'debloating' a product for unused components and containers?
-
P.2.2 Secure coding
Follow secure coding standards and practices appropriate to the development languages, APIs, AI models, and environment. Examples include validating all inputs, encoding outputs, and avoiding unsafe functions. Use development environments and static analysis tools that support compliance with secure coding practices as source code is being implemented.
Assessment questions
- What kind of secure coding practices are used?
- How are these practices communicated to developers and enforced?
-
P.2.3 Secure-by-default implementation
Reduce the likelihood of software deploying with weak security settings by defining secure deployment parameters so that the default settings are secure and do not weaken the security functions provided by the platform, AI models, network infrastructure, or services.
Assessment questions
- How do you determine whether the default installation settings for products, platform, models, network infrastructure, services defined to be secure by default?
-
P.2.4 Standard security features
Build support for standardized rather than proprietary security features, such as using existing log management, identity management, access control, or vulnerability management systems. These reused components are more likely to have their security posture already checked.
Assessment questions
- Please tell me about your philosophy of whether it is better to use available security features or to 'roll your own,' for example, log management, identity management, access control, and vulnerability management?
- What factors determine whether you use in-house or standard features?
-
P.2.5 In-house components
Well-secured in-house components and scripts are built following a secure SDLC process when third-party components cannot meet development needs. Similar to the processes for third-party components, in-house components should be kept in a repository, maintained, and regularly scanned for vulnerabilities, with new versions deployed through the organization as needed.
Assessment questions
- What processes are used to scan in-house components built using SDLC processes for vulnerabilities, and how is the process similar or different to what is done for third-party components?
- What causes a need for a new version?
- How are in-house components monitored to ensure they are regularly maintained?
-
P.2.6 Confirm Integrity of AI model data
Inputs to AI model training, evaluation, and the outputs of AI models should be evaluated for security risks, e.g. bias and data poisoning.
Assessment questions
- What processes are used to evaluate AI models build or consumed in your SDLC, and how is the process similar or different to what is done for third-party components?
- How are AI models monitored to ensure they are regularly maintained?
- How do you track the provenance of your AI models and data?
P.3 Manage component and container choices
-
P.3.1 Component and container choice
Consider component characteristics, such as the use of secure SDLC practice, evidence of ongoing maintenance, and open vulnerabilities, as quality indicators of direct and transitive dependencies. OpenSSF Scorecard metrics may be used as an indication of the security posture of a component. Components with the least functionality reduce the attack surface. Have a deny-list that prevents malicious components from being consumed.
Assessment questions
- What kind of approval process do you have, if any, for third-party libraries and containers included in a product?
- What do you consider when selecting a third-party component or container?
- Is considering solutions with the least functionality to reduce the attack surface part of your component selection process?
- How would the approval process handle dependencies that are no longer receiving updates?
- How do the OpenSSF Scorecard metrics factor in to decisions you make about components?
- Are you concerned when a component contains binary artifacts that cannot be easily reviewed and more easily sneak in a vulnerability?
- Do you have a Deny List to prevent the choice of vulnerable and malicious components?
-
P.3.2 Trusted repositories
Trusted public repositories may require signed packages and provide the means to verify the signatures; the packages and containers from these ecosystems should still be scanned. Organizations should host scanned repositories for high-assurance software and restrict build machines to only those sources.
Assessment questions
- Describe any guidelines you have for searching for candidate components in package managers trusted by your organization?
- How do you develop, publish, and maintain lists of container registries you trust?
-
P.3.3 Require signed commits
Using code that has been contributed with signed source code commits provides an integrity mechanism.
Assessment questions
- How do you require the software producers to sign code commits of the components and tools you bring into your product?
-
P.3.4 Vetted third-party component and container repositories
Components, binaries, containers, and AI models chosen from public ecosystems undergo a vetting process, including reviewing component meta-data and provenance data; secure composition analysis; binary composition analysis; and other security scanning. Rebuild software dependencies from source code when possible to remove additional software that may be contained in a compiled version. Establish repositories to host organization-approved components and containers.
Assessment questions
- Describe the vetting and scanning process you use when you choose components, containers, binaries and AI models from public ecosystems or from third-parties.
- How do you arrange for a local component repository in which binaries and source code of third-party components are stored?
- What are your policies for rebuilding components from source?
P.4 Discover vulnerabilities
-
P.4.1 Security code review
Perform peer code review on in-house developed code, AI-generated code, and some third-party code based on the organization’s secure coding standards. Create and use review checklists that may be informed by 'Top N' vulnerability lists. While the manual review of another programmer before check-in is essential for finding vulnerabilities injected with malicious intent, supplementing code review with automated tools, such as static analysis tools, is beneficial. Record and triage discovered issues and recommended remediations in the development team’s workflow or issue-tracking system.
Assessment questions
- How does the team perform security-related code reviews?
- Describe the secure coding standards your teams use, if any.
- How do discovered vulnerabilities get recorded, tracked, and remediated?
-
P.4.2 Automated security scanning tools
Specify which tools or tool types must or should be included in each toolchain to reduce human effort in discovering vulnerabilities and mitigating identified security risks and to improve the accuracy, reproducibility, usability, and comprehensiveness of security practices in the SDLC. Tools commonly include SAST, DAST, IAST, SCA, and secret detection tools. Specify how the toolchain components will be integrated, such as through the CI/CD pipeline. Follow recommended practices for tool selection/scanning, deploying, operating, and maintaining tools and toolchains.
Assessment questions
- What security tools are in your toolchain?
- SAST?
- DAST?
- IAST?
- How often is the scanning done for third-party components?
- Do you use a tool to detect secrets in code?
- What happens with the reported vulnerabilities?
- How do you scan your tools for vulnerabilities?
- How do you scan acquired AI models for security issues?
- How do you scan released AI models for security issues?
-
P.4.3 Automated vulnerability detection
Review, analyze, and/or test the software’s code, and the tools used to build it, including AI models, to identify or confirm the presence, remediation, and disclosure of previously undetected vulnerabilities. Tools commonly include SAST, DAST, IAST, SCA, and secret detection tools. Record and triage discovered issues and recommended remediations in the development team’s workflow or issue-tracking system.
Assessment questions
- How is code reviewed, analyzed, and tested to identify or confirm the presence, remediation, and disclosure of previously-undetected vulnerabilities such as through automated tools?
- How are the tools in the build and deploy pipelines reviewed, analyzed, and tested to identify or confirm the presence, remediation, and disclosure of previously-undetected vulnerabilities such as through automated tools?
-
P.4.4 Executable security testing
Executable testing, such as done by in-house testers or third-party penetration testers, is performed to find previously-undiscovered vulnerabilities. In-house testers should be guided by design review results, threat modeling, security requirements, and the security mechanisms in security features. Testing should include containers and AI models. Automate security tests as possible. Scope the testing, design the tests, perform the testing, and document the results, including recording and triaging all discovered issues and recommended remediations in the development team’s workflow or issue tracking system. Test the build process and scripts.
Assessment questions
- How are security features tested?
- How are test cases developed to test the security of an application?
- How is security testing factored into a product's test suite?
- How is executable code testing, such as in-house or external penetration testing, used to find vulnerabilities not found by previous reviews, analysis, and testing?
- How do you conduct executable testing on containers?
- How do you test your build process, scripts, GitHub actions?
-
P.4.5 Regular third-party compliance
Regularly check that third-party software components, including AI models, comply with the contractual requirements, such as following a secure SDLC, and fix detected vulnerabilities. Regularly scan components and containers to monitor for vulnerabilities and for evidence that artifacts are being maintained and have not been abandoned or deprecated.
Assessment questions
- How do you verify that third-party software components continue to comply with the requirements defined by the organization, such as through using a secure SDLC and the delivery of attestation of security practices?
- How do you check that components are being maintained and are not at end-of-life or have been abandoned?
- How are known critical vulnerabilities in third-party and open-source components and containers mitigated?
P.5 Manage vulnerable components and containers
-
P.5.1 SBOM consumption
Obtain or generate an SBOM for a product that provides a clear and direct link to the dependencies and their versions used within a product. Ideally, the SBOM will provide signed metadata from the build process. Tools can automate analysis and obtain desired information from the SBOM, such as aligning with vulnerability data to identify vulnerabilities within the exact package contents.
Assessment questions
- How are SBOMs checked for information about pending security vulnerabilities?
- How do you consume the SBOMs for your components?
- Describe your experiences with being aided by information provided by the SBOM?
- How do you feel about the assistance provided by tools to help you consume an SBOM?
- If not now, do you have plans to consume SBOMs in the future?
- How do you consider and trust the VEX information in the SBOM?
- What expectations do you put on the component producers for the vulnerabilities identified in the SBOM?
-
P.5.2 Dependency update
SCA tools, SBOM tools, and ecosystem tools (such as Dependabot) inform a product of vulnerabilities and provide an automated pull request for new versions of direct and transitive dependencies discovered vulnerable. Organizations need a process/strategy for updating dependencies - which may be manual or automated. Specify the software assets that require automated updates, defined from criticality/risk-based analysis. Rebuild, do not patch, containers with vulnerabilities.
Assessment questions
- What is your strategy for updating dependencies based on SCA, SBOM, or automated pull requests?
- What is your strategy for updating dependencies, such as through a tool such as Dependabot that automates PR?
- How does the project use tools to help update its dependencies e.g. Dependabot, RenovateBot?
- Describe the risk/criticality prioritiziation scheme used when updating assets. How are the software assets that require automated updates defined from a criticality/risk-based perspective?
- How are containers updated when a vulnerability is detected?
E.1 Safeguard artifact integrity
-
E.1.1 Safely store release artifacts
Securely archive the necessary files and supporting data (e.g., integrity verification information, provenance data, attestations, metadata) to be retained for each software release.
Assessment questions
- What files and supporting data (like integrity validation, provenance, configuration files, and metadata) do you securely archive and retain for each software release?
-
E.1.2 Version control
Store project artifacts, including but not limited to source code, executable code, infrastructure as code, AI models, and configuration-as-code, in a repository with restricted access using the principle of least privilege based on the nature of the artifact. Use version control to track and store all changes to this code with accountability to an authenticated individual account with access granted to personnel, tools, and services. As appropriate, sign or encrypt artifacts. Indefinitely retain change history.
Assessment questions
- Describe how code and other important project artifacts are stored, e.g., with version control with access granted to strongly-authenticated personnel using the principle of least privilege.
-
E.1.3 Multi-factor authentication (MFA)
Enforce multi-factor authentication (MFA) at the source code repository level by requiring a soft or physical token in addition to traditional passwords/credentials that are more likely to be compromised using techniques such as brute force guessing.
Assessment questions
- How is MFA mandated and enforced at the source code repository level?
- Describe the tokens that are used, e.g. are soft or physical tokens used?
- How is MFA enforced for endpoints?
-
E.1.4 Developer SSH key
Use SSH keys or SSH certificates for developers and agents in CI/CD pipelines to access source code repositories from their development tools rather than traditional passwords/credentials that are more likely to be compromised. Passphrases should be used to protect SSH keys.
Assessment questions
- How do you avoid using passwords, e.g., through the use of SSH keys, SSH certificates, or short-term rotated Access Tokens?
-
E.1.5 Branch protection
Branch protection settings to enforce security policies, such as requiring reviews, passing checks, preventing overwriting of history, and signed commits before code is accepted into the main branch. Branch protection can prevent forced pushings, overwriting history, and obfuscation of code changes. In high-risk, high-assurance environments, attribution of code changes, commit signing, and full attestation can be used with branch protection to prevent and detect complex attacks.
Assessment questions
- Describe your use of branch protection settings to enforce security policies, such as requiring reviews, passing checks, preventing overwriting of history, and signed commits before code is accepted into the main branch?
- In high assurance, high risk environments, is commit signing with full-attestation required?
-
E.1.6 Decommission assets
Live but abandoned systems may not get new security patches or be monitored for malicious activity, even though they may be vulnerable to new security threats and can be an attack vector. When a product or system has been declared end-of-life, decommission associated accounts, machines, data, keys, and passwords.
Assessment questions
- What decommissioning procedures are in place when a live product or system goes to end-of-life?
E.2 Safeguard build integrity
-
E.2.1 Release policy verification
A release policy should be maintained as a policy template and outline the required workflow while the software is developed, built, tested, and packaged to ensure the integrity, authentication, and auditability of a software product from initiation to end-user installation. A framework, such as in-toto, can produce meta-data during the build pipeline to enable attestation to the steps in the desired workflow. The meta-data can be analyzed to ascertain whether the steps in the workflow have occurred and to produce cryptographic guarantees by hashing and signing the inputs and outputs of steps in the pipeline.
Assessment questions
- How do you specify a templated build policy that specifies the required build workflow to ensure the integrity, authentication, and auditability?
- How do you verify the build policy in a cryptographically provable way, such as with in-toto?
-
E.2.2 Verify dependencies and environment
Analyze the integrity of build tools, components, and containers before bringing them into the pipeline. Validate the point of origin/provenance, checksums, and signatures in the downloading and ingesting processes. Request SBOM, provenance, and self-attestation to aid in achieving this goal.
Assessment questions
- How do you track packages back to their repositories?
- How do you verify the integrity (i.e. digital signature or hash match) and the provenance data of each component and container?
- How do you validate that the build environment's sources and dependencies come from a secure, trusted source of truth?
- What is your dependency version constraint strategy, e.g. how do you manage pinning and floating versions of your dependencies?
- How do you validate that the deployment environment's sources and dependencies come from a secure, trusted source of truth?
-
E.2.3 Defensive compilation and build
Determine which compiler, interpreter, AI training, and build tool features should be used to reduce vulnerabilities, such as producing compiler warnings for vulnerable code that are treated as errors; application of obfuscation techniques; approved configurations being used; verification of sources and manifests of dependencies; and approved tools configurations are available as configuration-as-code.
Assessment questions
- How do you use compiler, interpreter, and build tool features to detect vulnerabilities?
- Describe your system for preventing use of a vulnerable and/or malicious component in the build, e.g. a Deny List.
-
E.2.4 CI/CD hosting and automation
All CI/CD pipeline build steps should be automated with automation standardized across the enterprise via clear templated build pipelines that meet organizational standards. This automation should be treated as 'pipeline as code' with modifications being reviewed and should be immutable. The build runs on a hosted build platform that generates and signs the provenance.
Assessment questions
- How do you ensure reproducibility of your build processes, e.g. through defined build scripts, with the only manual step being to invoke the script?
- Do you store the build definition executed by the build service in a version control system fetched through a trusted channel with a trustworthy provenance chain?
- How do you ensure the build environment is reproducible, e.g.Does the build run on a hosted build platform?
- How are the build steps, sources, and dependencies declared up front with immutable references/use immutable artifacts stored in a local repository manager?
- How are infrastructure, build scripts, and GitHub Actions handled 'as code', such as with code review, scanning, testing, and version control?
- How are configuration changes reviewed?
- How are CI/CD pipelines standardized across the enterprise?
-
E.2.5 Secured orchestration platform
The build pipeline should be a series of hardened build steps implemented through a hardened container image stored within a secured repository and deployed through a hardened orchestration platform, such as Kubernetes. Orchestration processes can take advantage of built-in and added-on security features, such as checking for secrets and rollbacks to ensure that each deployed workload meets predetermined security requirements.
Assessment questions
- How are containers used throughout the build process?
- Describe your orchestration layer, e.g. Kubernetes, if present.
-
E.2.6 Reproducible builds
Follow recommended security practices to deploy, operate, and maintain tools and toolchains, including the reproducible build steps whereby identical input artifacts are rebuilt from source in a trusted build environment, non-determinism is eliminated, and the results can be cryptographically attested to be the same output in a bit-for-bit comparison.
Assessment questions
- Describe your stance on reproducible builds, e.g. do you practice reproducible builds on different protected environments and compare bit-for-bit results?
-
E.2.7 Build output
Write the output from the build process to separate storage from the inputs. A process separate from the build process should upload the artifact to the appropriate deployable repository.
Assessment questions
- How does the project store the build artifacts?
- How does the project build and publish official packages?
-
E.2.8 Hardened and isolated builds
Builds should run in an ephemeral and hermetic (isolated and sealed) environment with no parameters or network connectivity except to the hardened local repository of source code and dependencies and code signing infrastructure.
Assessment questions
- How do you ensure that the build service runs in an ephemeral environment, such as a container or VM, provisioned solely for the build?
- How do you ensure that the build service runs in an isolated environment free of influence from other build instances?
- How can the build output be affected by user parameters other than the build entry point?
- Is the build run with no network access (i.e., hermetic)?
E.3 Secure software development environment
-
E.3.1 Authentication
Authenticate employees and contractors to enable access control, traceability, non-repudiation, chain-of-custody, and provenance for systems, software, and services.
Assessment questions
- How are employees and contractors authenticated to enable access control, traceability, non-repudiation, chain of custody, and provenance for systems, software, and services?
- Where do you store access credentials (e.g., hashes for passwords) and secrets in a secured (e.g., encrypted) location, such as a secure vault?
-
E.3.2 Environmental separation
Using network segmentation and access control to separate and protect each environment (e.g., endpoints, development, build, test, deployment, model training) involved in software development to protect from internal and external threats. Remote access to the build environment should be rare, logged, and require multi-party approval. Log and monitor access, particularly privileged access. Configure the environment's hosting infrastructure following a zero-trust architecture.
Assessment questions
- How are environments involved in software development (e.g., endpoints, development, build, test, model training, deployment) separated and protected from physical and remote access to the build environment?
- How is access logged?
- How are approvals arranged for access to the build environment?
- How are the build environment workstations held to a security standard, e.g. up-to-date patches, anti-virus definitions, etc?
-
E.3.3 Role-based access control
Define a set of roles and accounts with an associated access control/authorization level considering remote access requirements, the period for which access is needed, and careful vetting of contractor personnel. These roles can be used for access control to ensure proper traceability of actions and actors. A process should be implemented to manage mission-critical systems' temporary or emergency access. Role-based access is enforced by physical and logical access enforcement mechanisms.
Assessment questions
- How are authorization levels for accounts associated with roles and the associated level of access control, including remote access, proper vetting of contractors, and that authorization does not exceed the period of performance?
- How are access control violations handled?
-
E.3.4 Information flow enforcement
Specify how information flow across trust boundaries is enforced to ensure only the required information is communicated to participants in the supply chain, including but not limited to suppliers, developers, system integrators, and external system providers. Requirements for transmission confidentiality and integrity are integrated into agreements with suppliers. The degree of protection should be based on the sensitivity of the information. Security mechanisms such as authentication, authorization, and encryption can be used to achieve enterprise confidentiality and integrity requirements.
Assessment questions
- How is information flow enforced to ensure only the required information is communicated across trust boundaries to participants in the supply chain through mechanisms that ensure transmission confidentiality and integrity?
- What kind of requirements for transmission confidentiality and integrity are integrated into agreements with suppliers?
-
E.3.5 Baseline configuration
Establish a baseline configuration of the information system and the development environment, including documenting, formally reviewing, and securing the agreement of stakeholders. The baseline provides a starting point for tracking changes to components, code, and settings for traceability and provenance.
Assessment questions
- How is a baseline configuration of the information system and development environment initialized to provide a starting point for tracking changes to components, code, and/or settings for traceability and provenance?
- Who sets the baseline?
-
E.3.6 Monitor changes to configuration settings
Monitor and audit configuration settings and change controls within the information systems and networks throughout the SDLC. Changes should be tested and approved before being implemented. Configuration settings should be monitored so designated employees can be alerted when a change has occurred.
Assessment questions
- What kind of monitoring and auditing of changes to configuration settings is in place?
- What designated employees are alerted when an unauthorized change occurs?
-
E.3.7 Boundary protection
The enterprise should consider the trust boundaries and provide separation and isolation of development, test, and security assessment tools and external networks or information systems. The connections should only be through managed interfaces consisting of boundary protection devices arranged in accordance with an organizational security architecture.
Assessment questions
- How are development, test, and security assessment tools and operational environments separated and isolated from external networks or information systems and each other, as appropriate considering trust boundaries?
- How are trust boundaries considered, and how is the separation/isolation implemented?
- How is malicious behavior detection implemented and managed?
-
E.3.8 Key rotation
Have a key rotation policy to ensure that compromised keys will cease to be usable after a certain period. Replace and revoke a private key immediately if it is known to have been compromised to shut off access.
Assessment questions
- What is your key rotation policy?
-
E.3.9 Ephemeral credentials
As an alternative to passwords, ephemeral credentials are randomly generated, short-lived access credentials that exist only for one session to authenticate and authorize privileged connections. These credentials are automatically issued as needed, so users do not have to input credentials when connecting and enable fine-grained permissions and automation of provisioning access tokens.
Assessment questions
- Describe your approach to access management for machines and services such as CI/CD pipeline agents, e.g. do you use short-lived tokens.
- Who uses these tokens and for what types of systems?
-
E.3.10 Establish a root of trust
Roots of trust (RoT) are highly-reliable hardware, firmware, and software components that perform specific, critical security functions, such as storing the keys used for cryptographic functions and enabling a secure boot process. Many RoT are implemented in hardware so that malware cannot tamper with the functions they provide. The RoT is established from an offline source.
Assessment questions
- Describe your root of trust for performing specific, critical security functions such as storing keys for cryptographic functions or enabling secure boot. How are these implemented?
D.1 Respond to/disclose vulnerabilities
-
D.1.1 Vulnerability analysis
Enter discovered vulnerabilities into a defect management system. Analyze each vulnerability to gather sufficient information to plan its remediation. Plan a risk response and prioritization, such as by estimating the probability of exploitation and the impact if it is exploited.
Assessment questions
- How are discovered vulnerabilities recorded and triaged?
- How are the vulnerabilities prioritized for remediation?
- What mechanisms are used for prioritization, e.g. is a severity rating, such as CVSS, used to aid prioritization?
- Is the access control to vulnerability information more restrictive than plain bug tracker defects?
-
D.1.2 Risk-based vulnerability remediation
Make risk-based decisions on whether to remediate a vulnerability or if the risk will be addressed another way (e.g. acceptance, temporary remediation, deferred remediation). Prioritize actions that will be taken. Deliver remediations via an automated and trusted delivery mechanism.
Assessment questions
- Tell me about your process for making decisions on which vulnerabilities should be remediated?
- What is your process for responding to reported incidents quickly?
- Do you have a policy that high-priority vulnerabilities can prevent the shipment of the product?
-
D.1.3 Vulnerability disclosure
Have a policy that invokes vulnerability disclosure and remediation, and organizational response. Implement the roles, responsibilities, and processes needed to support that policy, including a Product Security Incident Response Team (PSIRT) to handle responses to vulnerability reports and incidents. The disclosure program will require insight from internal stakeholders, such as legal, marketing, and public relations.
Assessment questions
- What is your process for addressing vulnerability disclosure and remediation with roles, responsibilities, and processes in place?
- How do you develop and release security advisories to your software acquirers?
- How do you publish a security policy (e.g. via a security.md file) to inform users about a vulnerability and how to report it?
- How do users know how to report vulnerabilities?
- Who handles PSIRT?
-
D.1.4 Vulnerability eradication
Review the software for similar vulnerabilities to eradicate a class of vulnerabilities, not just the instance originally discovered. Proactively fix a class of vulnerabilities rather than waiting for the discovery of each vulnerability through automation, testing, or an external incident. Automation and custom rules in vulnerability discovery tools or compilers can be used. Patterns of remediation efforts should provide feedback into the secure SDLC.
Assessment questions
- Describe the team's process of proactively eradicating or reducing a whole class of vulnerabilities based upon previously-identified vulnerabilities or incidents?
- How are these patterns folded back into the SDLC?
-
D.1.6 Root cause analysis
Analyze vulnerabilities discovered throughout the process and in production to determine their root cause for being injected and not discovered earlier before production. Incident response feedback should be fed back to developers and be considered when evolving the SDLC. Record lessons learned. Identify patterns, such as security controls in the SDLC needing to be followed. Discuss and disseminate these patterns to developers. Guide process correction if the current process is not being followed.
Assessment questions
- How do you analyze identified vulnerabilities to determine their root cause, including how the vulnerability was injected and, perhaps, how it escaped vulnerability detection efforts and made it into production?
- How do you communicate trends to developers? How can trends identified in root cause be factored into your SDLC process?
D.2 Monitor intrusions/violations
-
D.2.1 System monitoring
Continuously monitor the running system to gather information for risk decisions, criticality analysis, vulnerability and threat analysis, incident response, policy non-compliance, and insider threat detection, including boundary protection, supply chain components, and supply chain information flow.
Assessment questions
- How are running systems continuously monitored to gather information for risk decisions, criticality analysis, vulnerability and threat analysis, incident response, and insider threat, including boundary protections of supply chain components and supply chain information flow?
- What is the process when an intrusion or violation is detected? Is there a QoS target?
-
D.2.2 Build process monitoring
Do you monitor the build chain for unauthorized access and modifications? Have you completed an attack surface investigation of your build environment? What kinds of actions do you take to narrow the attack vectors?
Assessment questions
- How do you monitor the build chain for unauthorized access and modifications?
- How recently have you completed an attack surface investigation of your build environment?
- What kinds of actions do you take to narrow the attack vectors?
Your radar chart
average score per practicePractices with no scored controls sit at the centre and are marked "not scored".