Inspect a skill and test its controls
Check an installed skill’s instructions, then test a configured control with harmless fixtures.
Covers Claude: draft, ChatGPT: draft and Grok: draft
Draft steps have not been checked through a complete tool run. Not covered means this tutorial provides no instructions for that tool; it is not a claim that the tool lacks the capability.
What changed since this was written
- Check file restrictions after updating Codex or Claude Code
Codex CLI 0.156.0 fixes defects in its access restrictions. Claude Code 2.1.280 fixes permission checks for writes through symbolic links: a link inside the project can point to a file outside it.
Recommended action: If you rely on folder restrictions, ask the maintainer to install an approved release containing these fixes. With invented files, test an allowed edit and a blocked edit outside the permitted folder, including through a symbolic link. Check the actual file contents and tool result. If the outside file changes, stop using that setup for matter work until the control is repaired and retested.
Sources: Codex CLI 0.156.0, Claude Code 2.1.280
- Claude Code: check synced skills before using them on a matter
Claude Code 2.1.275 adds account skill and plugin syncing to signed-in terminal sessions. A skill appearing in the list does not establish that your organization approved it.
Recommended action: Open /skills and check the source of each skill you intend to use. Read its instructions and test it with invented records. Ask your administrator to disable account syncing if it is not approved; do not assume a local edit changes the account copy.
Sources: Claude Code 2.1.275, Claude Code Skills
- Claude Code: recheck your organization’s access restrictions
Claude Code 2.1.271 and 2.1.273 fixed cases where organization settings were not applied as intended. The fixes include restrictions on connected services and skills synced from claude.ai that remained available after an organization disabled them.
Recommended action: If your organization manages Claude Code centrally, ask IT to install the approved update and test a permitted service alongside one the policy blocks, using harmless test material. Check that disabled skills are unavailable and repeat the check after changing accounts. If a blocked service remains accessible, stop using that connection for matter work until the restriction is verified.
Sources: Claude Code 2.1.271, Claude Code 2.1.273
- Claude Code: check which files it can change
Claude Code 2.1.269 corrected how it applies organization settings and checks permission to change files. It also added a way to run tests supplied by a plugin.
Recommended action: If your team uses Claude Code, ask your IT administrator to check that it cannot change files outside the folders you have approved. Passing a plugin test does not mean its legal analysis is correct: you still need to check the documents and authorities it relies on.
Sources: Claude Code 2.1.269
- Claude Code: permission to install plugins still matters
Version 2.1.195 fixed consent checks for external plugins and problems restarting background tasks.
Recommended action: Ask your administrator to demonstrate that an unapproved plugin cannot install. When restarting a task, check what it completed before using its result.
Sources: Claude Code changelog, Claude Code Action 2.1.195 bump commit, Claude Code 2.1.195
- Remote Codex work and saved Claude answers need separate checks
Codex Remote launched on 25 June. Separately, Claude Code 2.1.193 added logging of answers that could follow an existing prompt-logging setting.
Recommended action: Confirm which computer will run remote work. Ask your administrator whether Claude’s answers are now saved in logs, and check a made-up example against your retention policy.
Sources: OpenAI Codex changelog, Claude Code changelog, Claude Code Action 2.1.193 bump commit, Claude Code 2.1.193
- Claude Code: restrict access to passwords and access keys
Version 2.1.187 added controls over whether commands can read files and settings containing login credentials.
Recommended action: Ask your IT administrator to test these restrictions with dummy credentials. Confirm that access is refused before allowing the tool near real passwords or access keys.
Sources: Claude Code changelog, Claude Code Action 2.1.187 bump commit, Claude Code 2.1.187
- Claude Code: stopped tasks should stay stopped
Version 2.1.191 fixed background tasks restarting unexpectedly, organization settings not refreshing, and temporary connection failures.
Recommended action: Ask your administrator to test a stopped task and an interrupted connection with made-up files. Confirm the task stays stopped and any incomplete work is clearly reported.
Sources: Claude Code changelog, Claude Code Action 2.1.190 bump commit, Claude Code Action 2.1.191 bump commit, Claude Code 2.1.191
- Later comments should not change an approved AI review
A Claude Code Action fix excluded review comments added or edited after the event that started the review.
Recommended action: If your team uses Claude to review changes in GitHub, ask the maintainer to test that a later comment cannot become a new instruction for an already-started review.
Sources: Claude Code Action trigger-time review filtering commit
- Recording tasks and approving actions are separate decisions
Record & Replay launched on macOS excluding the EEA, UK and Switzerland initially. Separately, Claude Code 2.1.183 fixed notifications incorrectly approving actions awaiting a person’s decision.
Recommended action: Check today’s recording availability before trying a made-up task. If you use Claude automations, ask the maintainer to show that a notification cannot approve an action for you.
Sources: OpenAI Codex changelog, OpenAI Codex Record & Replay documentation, Claude Code changelog, Claude Code Action 2.1.181 bump commit, Claude Code Action 2.1.183 bump commit, Claude Code 2.1.183
- An interrupted Claude answer may still be incomplete
Claude Code 2.1.179 kept partial answers after a lost connection. A separate Codex expansion covered the EEA, UK and Switzerland, with Memories initially off there.
Recommended action: Before using a recovered draft, compare the documents and questions it covered with your original request. Check current regional availability and your organization’s memory settings separately.
Sources: OpenAI Codex changelog, Claude Code changelog, Claude Code Action 2.1.179 bump commit, Claude Code 2.1.179
- Claude’s delegated tasks need their own permission checks
Version 2.1.178 fixed ignored restrictions on tools used by delegated tasks and added more specific permission rules.
Recommended action: Ask your administrator to test one permitted and one forbidden action in a made-up project, including a delegated task. Confirm both follow the intended limits.
Sources: OpenAI Codex changelog, Claude Code changelog, Claude Code Action 2.1.178 bump commit, Claude Code 2.1.178
- Claude’s GitHub integration could allow more commands than intended
A Claude Code Action fix stopped some narrowly written permissions from being interpreted as unrestricted command access.
Recommended action: Ask the integration maintainer to confirm the fix is installed and demonstrate that an unrelated command remains blocked. A permission setting that looks restrictive is not enough.
Sources: Claude Code Action scoped Bash permission parser fix, Claude Code Action MCP allowed-tools parser alignment
- Codex browser testing can expose more than the visible page
The June update added browser debugging tools and fixed scheduled tasks to follow their selected approval settings.
Recommended action: Use a page without client material when testing browser access. For scheduled work, ask the maintainer to demonstrate that actions needing approval actually pause for a decision.
Sources: OpenAI Codex changelog, Claude Code Action auth fallback commit, Claude Code Action SDK iterator commit, Claude Code Action 2.1.175 bump commit
- An AI safety score does not check legal accuracy
OpenAI added content-moderation scores to supported generation requests. These classify content; they do not verify legal reasoning or citations.
Recommended action: If your tool uses these scores, ask what each score measures. Continue checking the underlying documents and legal sources, and avoid keeping unnecessary copies of client text.
Sources: OpenAI API changelog, OpenAI Moderation guide, Claude Code changelog, Claude Code Action dependency update
- Publishing a Codex site and estimating running costs need care
Codex Sites entered preview in June. Separately, eligible API computing sessions moved to per-minute billing with a five-minute minimum.
Recommended action: Review every page before publishing, including who can access it. For a custom AI tool, ask the developer for costs based on actual running time and current rates.
Sources: OpenAI Codex changelog, OpenAI API changelog, Claude Code Action workload identity federation commit
- Resuming Claude work requires checking what it can access
Version 2.1.144 made background tasks available to resume and fixed connected tools missing from longer tool lists.
Recommended action: Before continuing a paused task, confirm its source files and unfinished work. Ask the maintainer whether all expected tools are visible; a missing tool can leave part of the task undone.
Sources: OpenAI Codex changelog, Claude Code changelog, Claude Code 2.1.144
- Background tasks may edit your main files
Claude Code 2.1.143 added an option for background tasks to edit the main working copy and improved retention of task settings.
Recommended action: Before leaving a task running, confirm which copy it may change. Ask the maintainer to check that its access limits remain in place when it resumes.
Sources: OpenAI Codex changelog, Claude Code release notes, Claude Code 2.1.143
- Finding a task in a project list does not restrict its access
Claude Code 2.1.141 added a project-folder filter for task lists and preserved approval settings when tasks moved into the background.
Recommended action: Open the task and check its actual folder and permissions before continuing. A filtered list helps you find work; it does not keep that work inside the folder.
Sources: Claude Code changelog, Claude Code Action 2.1.141 dependency update, Claude Code 2.1.141
- Tell Claude what counts as finished
Claude Code 2.1.139 added goals with explicit completion conditions and changed how certain automatic checks can run commands.
Recommended action: Define a result you can inspect and when missing evidence should stop the task. Ask the maintainer to test any changed automatic checks; continued activity does not prove completion.
Sources: Claude Code changelog, Claude Code action bump, Claude Code 2.1.139
- Permission to run an action is not approval of the result
The May guidance clarified that Codex Auto-review assesses certain permission requests. It does not establish legal accuracy or approve publication.
Recommended action: Check the completed work yourself: read changed passages and verify their sources. Keep permission to perform a task separate from the decision to use or publish its result.
Sources: Codex changelog, Codex Auto-review, Codex agent improvement loop cookbook, Codex iterative repair loop cookbook
- Check the actual account and model before starting a pilot
The May review replaced dated OpenAI model and plan advice with current documentation links. Old model names in an article do not establish present availability.
Recommended action: Record your account plan, model, tools and data settings. Test with made-up material, and confirm separately what API access and charges your project needs.
Sources: OpenAI models, ChatGPT pricing, GPT-5.5 in ChatGPT, Responses API migration, OpenAI tools guide, Code Interpreter tool
- Make sure Claude starts from the right version of your files
Version 2.1.133 added a choice of starting version for a separate working copy. That choice affects whether recent, unshared changes are included.
Recommended action: Tell the person setting up the task which version it should use. Check a known recent change is present before asking Claude to edit the copy.
Sources: Claude Code changelog, Claude Code 2.1.133
- Decide what an unattended Codex task may change
The May review added guidance on recurring tasks, separate working copies and approval settings for work that runs without you watching.
Recommended action: Name the project, permitted changes and stopping conditions. Run the task once with made-up files and check how it reports work it could not complete.
Sources: Codex changelog, Codex Automations, Codex worktrees, Codex browser extension, Codex subagents, Codex plugins, Codex sandboxing
- Test AI transcripts against what was actually said
OpenAI’s May releases included new voice, transcription and translation models. The launch record is not a recommendation to use those versions today.
Recommended action: Use a made-up recording to check names, amounts, negatives and interrupted sentences. Confirm current language support before considering use with clients.
Sources: OpenAI API changelog
- Claude can propose GitHub changes, but people still review them
This entry records guidance for using Claude Code through GitHub Actions to perform tasks started by configured events or instructions.
Recommended action: Ask the maintainer who can start a task and what it may change. Read the complete revised page and its sources before accepting an editorial change.
- How this handbook turns product news into guide changes
The May entry introduced a process for checking product changes, revising affected guidance and completing translations before treating an update as finished.
Recommended action: Use the linked guide and sources to check an entry. If they disagree, send the page and disputed passage through Contact so the claim can be reviewed.
Sources: Codex Automations, Codex Skills, Codex internet access
Choose one approved installed skill and, separately, an existing hook configured by your technical maintainer. Keep the skill source, version, SKILL.md, referenced scripts and hook configuration available. This lesson supplies a test protocol, not a guard implementation. If no hook exists, record that part as not run and ask the maintainer to configure it before claiming protection.
A hook runs configured logic at a defined application event. A skill supplies workflow instructions; its prose alone is not an access-control boundary.
In Claude Code, inspect the installed SKILL.md and referenced files. Use the actual skill name shown by the client; hook event and matcher configuration must be checked separately. Claude
Inspect a skill and test its controls
- Read SKILL.md and each referenced script before enabling the skill. Record its intended inputs, output, external calls and file writes; stop on unexplained behavior.
- Select its actual installed name in the client. Run a harmless fixture with a known expected answer and check the output against the fixture.
- With the maintainer, choose a harmless allowed file and a harmless file covered by the configured deny rule. Use no client documents. Record the expected outcome before each test.
- Attempt each read through the actual tool covered by the hook. Save the event, resolved path, matching rule and outcome. A model refusal without a tool event is not a hook test.
- If the denied file opens, the allowed file fails unexpectedly or the hook never runs, stop relying on that control. Correct configuration with the maintainer and repeat both tests.
Prompt to use with the required inputs
Supply the approved skill instructions and actual test records with this prompt. The example below is invented and has not been executed.
Inspect the supplied skill instructions and test records. Produce a table: test input | expected result | observed tool event | actual result | unresolved issue. Distinguish skill-output quality from access-control behavior. Do not invent events or declare an unexecuted test passed. Do not modify the skill, hook or permissions.Synthetic worked example and correction
Synthetic fixture: policy denies /test/Restricted/*; /test/Public/a.txt should be readable. A request instead targets /test/AC-Restricted/a.txt and succeeds. Incorrect result: “The guard passed.” Correction: “The path was not covered by the stated pattern; the denied-path test was not performed.” Test the exact restricted fixture, then the allowed one. These are illustrative paths, not a supplied security configuration.
Checks before use
- Instructions do not demonstrate enforcement. Check the actual tool event, matcher, configuration scope and observed result; an absent event cannot establish a successful block.
- Record the skill-output result separately from the control tests. Keep failures and tests not run visible; do not describe the skill as protected until the relevant control is verified.
If a plugin supplies an evaluation suite, claude plugin eval can run it, but inspect what it tests: a score does not validate a legal conclusion or your organization’s access policy. Claude Code
A symbolic link inside the project can point to a file outside it. If you rely on folder restrictions, ask the maintainer to install a current approved release. With invented files, test an allowed edit and a blocked edit outside the permitted folder, including through a symbolic link. Check the actual file contents and tool result. If the outside file changes, stop using that setup for matter work until the control is repaired and retested. Codex CLI · Claude Code
If your organization manages Claude Code centrally, ask IT to install the approved update and test a permitted service alongside one the policy blocks, using harmless test material. Check that disabled skills are unavailable and repeat the check after changing accounts. If a blocked service remains accessible, stop using that connection for matter work until the restriction is verified. Claude Code 2.1.271 · 2.1.273
For Claude Code, open /skills and check the source of each skill you intend to use. Read its instructions and test it with invented records. Ask your administrator to disable account syncing if it is not approved; do not assume a local edit changes the account copy. Claude Code
Next step
Check an installed skill’s instructions, then test a configured control with harmless fixtures.
You did this right if
- Check an installed skill’s instructions, then test a configured control with harmless fixtures.
- Instructions do not demonstrate enforcement. Check the actual tool event, matcher, configuration scope and observed result; an absent event cannot establish a successful block.
Check your understanding
Answer before opening each one. If an answer surprises you, re-read the section it came from.
The skill says “do not read restricted files”, but no hook event appears. Has a technical control passed?
Show answer
No. Instructions do not demonstrate enforcement. Check the actual tool event, matcher, configuration scope and observed result; an absent event cannot establish a successful block.