Commit Graph

487 Commits

Author SHA1 Message Date
ChristopherHX
3c7eda7f3e fix: docker stub and add a test for this (#2355)
* fix: docker stub

* test if you can build run without docker

---------

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2024-07-10 16:45:07 +00:00
ChristopherHX
6de25a53bc fix: make node tool non volatile (#2372)
* fix: make node tool non volatile

Currently downgrading node via setup-node can break later actions

* fix it and lookup on startup

* fix problems

---------

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2024-07-10 15:58:37 +00:00
ChristopherHX
2ad5ff74f8 fix: add missing service container health check (#2354)
* fix: Implement missing health ceck for Services

* Add test case

* linter doesn't support min builtin and fix check

---------

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2024-07-10 15:33:54 +00:00
Louis Auzuret
e4607fc791 fix: skip service container for empty image (#2281)
* fix: skip service container for empty image

It is used to skip empty image name in services which is the only way to handle condition services in github action currently https://github.com/actions/runner/issues/822

* test: add testdata for empty image in services

* fix: add missing test call

* fix: wring test call

* fix: invalid without expression

---------

Co-authored-by: ChristopherHX <christopher.homberger@web.de>
2024-06-05 19:16:34 +00:00
ChristopherHX
b5ad3c4acd fix: composite action input pollution (#2348)
* fix: composite action input pollution

* fix run steps

* fix missing defaults in post after env cleanup

* fix test to make more sense

* Add tests and simplify change

---------

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2024-06-05 14:44:44 +00:00
ChristopherHX
b917ecc184 fix: update reusable workflow input handling (#2349)
* update reusable workflow input handling

* make test stricter

---------

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2024-06-05 14:30:12 +00:00
ChristopherHX
4977ba910c Meta-copy-symlinks-new-actions-cache (#2337)
* meta: copy symlinks

* copy file (old style)

* copy files (new styles)

* avoid spelling error detection

* add error

* fix

---------

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2024-05-29 00:12:32 +00:00
ChristopherHX
e1e5671e3d Actions Artifacts v4 backend (#2224)
* Actions Artifacts v4 backend

* lint

* fix it now

* remove protofile, to make linter not complain

* sync changes

* add delete

* import auth test from gitea

* add more tests

* codecov ignore protobuf
2024-05-20 20:00:04 +00:00
Andreas Taylor
657a3d768c Fix for issue 2232: Many lines of "Could not find any stages to run" on run (#2272)
* Initial commit

* Put the tests back

* Remove unnecessary checks

* Remove unneeded check and fix test code

---------

Co-authored-by: Jason Song <i@wolfogre.com>
2024-05-13 15:41:28 +00:00
Yaroslav Halchenko
843cd94cc2 Add codespell support (config, workflow to detect/not fix) and make it fix few typos (#2296)
* Add github action to codespell master on push and PRs

* Add codespell config with few custom skips

* [DATALAD RUNCMD] Do interactive fixing of  leftover ambigous typos

=== Do not change lines below ===
{
 "chain": [],
 "cmd": "codespell -w -i 3 -C 2 ./pkg/common/git/git.go",
 "exit": 0,
 "extra_inputs": [],
 "inputs": [],
 "outputs": [],
 "pwd": "."
}
^^^ Do not change lines above ^^^

* exclude pkg/runner/hashfiles/index.js

* [DATALAD RUNCMD] run codespell throughout fixing typos automagically

=== Do not change lines below ===
{
 "chain": [],
 "cmd": "codespell -w",
 "exit": 0,
 "extra_inputs": [],
 "inputs": [],
 "outputs": [],
 "pwd": "."
}
^^^ Do not change lines above ^^^

---------

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2024-04-19 09:22:45 +00:00
dependabot[bot]
2a2cd7d5bc build(deps): bump github.com/moby/buildkit from 0.12.5 to 0.13.1 (#2253)
* build(deps): bump github.com/moby/buildkit from 0.12.5 to 0.13.1

Bumps [github.com/moby/buildkit](https://github.com/moby/buildkit) from 0.12.5 to 0.13.1.
- [Release notes](https://github.com/moby/buildkit/releases)
- [Commits](https://github.com/moby/buildkit/compare/v0.12.5...v0.13.1)

---
updated-dependencies:
- dependency-name: github.com/moby/buildkit
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* update error messages

* remove point

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: ChristopherHX <christopher.homberger@web.de>
2024-04-11 22:39:17 +00:00
Jason Song
2df924c509 Bump docker to v26 (#2289)
* chore: bump docker

* chore: bump more

* chore: update codes
2024-04-11 08:29:02 -07:00
ChristopherHX
5a9dd3d0ea fix: skipped jobs have no result (#2276)
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2024-04-11 02:31:18 +00:00
lvyaoting
ac13a1b20d chore: fix some comments (#2279)
Signed-off-by: lvyaoting <lvyaoting@outlook.com>
Co-authored-by: ChristopherHX <christopher.homberger@web.de>
2024-04-10 19:58:56 +00:00
Jason Song
cdc22da7f0 Remove local action cache if remote has changed (#2284)
* fix: remove local cache if remote is changed

* test: TestCloneIfRequired
2024-04-10 12:41:02 +00:00
ChristopherHX
f825e42ce2 fix: cache adjust restore order of exact key matches (#2267)
* wip: adjust restore order

* fixup

* add tests

* cleanup

* fix typo

---------

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2024-03-29 02:07:20 +00:00
Jason Collins
d9a19c8b02 Trivial: reduce log spam. (#2256)
Co-authored-by: ChristopherHX <christopher.homberger@web.de>
2024-03-28 23:28:48 +00:00
James Kang
3949d74af5 chore: remove repetitive words (#2259)
Signed-off-by: majorteach <csgcgl@126.com>
Co-authored-by: ChristopherHX <christopher.homberger@web.de>
2024-03-28 23:14:53 +00:00
Jason Song
b9382a2c4e Support overwriting caches (#2265)
* feat: support overwrite caches

* test: fix case

* test: fix get_with_multiple_keys

* chore: use atomic.Bool

* test: improve get_with_multiple_keys

* chore: use ping to improve path

* fix: wrong CompareAndSwap

* test: TestHandler_gcCache

* chore: lint code

* chore: lint code
2024-03-28 16:42:02 +00:00
Jason Song
f56dd65ff6 test: use ping to improve network test (#2266) 2024-03-28 11:56:26 +00:00
ChristopherHX
119ceb81d9 fix: rootless permission bits (new actions cache) (#2242)
* fix: rootless permission bits (new actions cache)

* add test

* fix lint / more tests
2024-03-08 01:25:03 +00:00
huajin tong
352ad41ad2 fix function name in comment (#2240)
Signed-off-by: thirdkeyword <fliterdashen@gmail.com>
2024-03-06 14:20:06 +00:00
ChristopherHX
75e4ad93f4 fix: docker buildx cache restore not working (#2236)
* To take effect artifacts v4 pr is needed with adjusted claims
2024-03-05 06:04:54 +00:00
ChristopherHX
ac6a465e27 feat: Test writing remote Actions locally (#2155)
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2024-02-23 07:40:14 +00:00
ChristopherHX
5601fb0e13 refactor: simpilfy go-git cache (#2208) 2024-02-18 03:53:22 +00:00
ChristopherHX
852959e1e1 feat: offline mode for new action cache (#2173)
* Try fetch update of the action, otherwise use cached version

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2024-02-12 12:51:37 +00:00
Kristofer
f2e65e1d40 fix: Rootless error concerning /var/run/docker.sock (#2181)
* Use same socket defaulting strategy every time

* Always default to DOCKER_HOST

* Add more debug logs

* Commenting, and massively simplified socket logic

* Rever to upstream run_context.go

* Fix EACCESS error regarding /opt/hostedtoolcache

* Revert "Fix EACCESS error regarding /opt/hostedtoolcache"

This reverts commit b2a8394d3358e1b5aab9dabe555d4a3f2bf0b2f9.

* Revert CLI debug logs

* Move socket and host handling to own function, and simplify logic

* Move to container package

* Make return be a struct

* Write tests to verify functionality

* Fix DOCKER_HOST being set to the string "DOCKER_HOST"

* Always use struct

* Use socketLocation, for DOCKER_HOST and more defaults

* Fixup arguments to GetSocketAndHost in test and root.go

* Un-struct hasDockerHost

* Fixup logic and set hasDockerHost

* Minor scoping & variable name change

* Move functionality to a new file

* Rename corresponding test

* Reviewfix

* Fix DOCKER_HOST expected

* Fix test assertions and add comments

* Swap comparison actual, expected

* Fixed no-DOCKER_HOST env test

* Fixed default socket test

* Add test to verify review comments

* Add more test for greater test coverage

* Consistent comment references

* Fix bug found while writing tests

* Passing tests

* NoMountNoHost testfix

* Rename test appropriately

* NoMount testfix

* Fixed OnlySocket

* Swap expected <-> actual in tests

---------

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2024-02-06 17:18:11 +00:00
Markus Wolf
054caec791 fix: use correct path to toolcache (#1494)
The toolcache on GitHub Actions need to be in
/opt/hostedtoolcache. This is the case for all
environment variables set by act, but it's not the
case for the volume mounted into the container.

Co-authored-by: Björn Brauer <zaubernerd@zaubernerd.de>
Co-authored-by: ChristopherHX <christopher.homberger@web.de>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2024-01-30 22:43:52 +00:00
ChristopherHX
5a80a044f9 refactor: filecollector into new package (#2174)
* refactor: filecollector into new package

* Add test for symlinks

* add test fix bug of GetContainerArchive

* add test data
2024-01-30 00:46:45 +00:00
Josh Soref
5e0d29d665 fix: improve warning about remote not found (#2169)
Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2024-01-28 19:21:21 +00:00
ChristopherHX
6dd67253bc fix: improve new-action-cache fetch failure error (#2172)
- include repoURL and repoRef in error
- map NoErrAlreadyUptodate to `couldn't find remote ref` for branchOrtag
  fetch request

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2024-01-28 17:02:15 +00:00
ChristopherHX
09d4b5d6ad fix: subpath actions via new artifact cache (#2170)
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2024-01-28 16:49:47 +00:00
ChristopherHX
a6ec2c129a fix: improve action not found error (#2171) 2024-01-28 16:37:19 +00:00
Matthew
6a8c42ac53 Add containerd's normalized architectures to archMapper (#2168) 2024-01-24 02:44:48 +00:00
胖梁
8072a00a77 WorkflowDispatchConfig supports multiple yaml node kinds (#2123)
* WorkflowDispatchConfig supports ScalarNode and SequenceNode yaml node kinds

* Avoid using log.Fatal

* package slices is not in golang 1.20

---------

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2024-01-20 14:07:36 +00:00
Kristoffer
7f7d84b10f fix: match cache restore-keys in creation reverse order (#2153)
* Match cache restore-keys in creation reverse order

* Match full prefix when selecting cache

---------

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2024-01-20 12:11:50 +00:00
TKaxv_7S
f055d4ae60 feat: support offline mode (#2128)
* Add: Actions Offline Mode

* Add: Actions Offline Mode

---------

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2024-01-20 00:20:15 +00:00
ChristopherHX
f7a846d2f5 feat: cli option to enable the new action cache (#1954)
* Enable the new action cache

* fix

* fix: CopyTarStream (Docker)

* suppress panic in test

* add a cli option for opt in

* fixups

* add package

* fix

* rc.Config nil in test???

* add feature flag

* patch

* Fix respect --action-cache-path

Co-authored-by: Björn Brauer <zaubernerd@zaubernerd.de>

* add remote reusable workflow to ActionCache

* fixup

---------

Co-authored-by: Björn Brauer <zaubernerd@zaubernerd.de>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2024-01-19 23:49:35 +00:00
Leonardo Taccari
adbe229fcb Add support for NetBSD (#2023)
NetBSD can run Docker CLI and then use Docker on some remote machine
via DOCKER_HOST.

(This can be probably extended to all other Unix-es capable of running
just Docker CLI.)

Co-authored-by: ChristopherHX <christopher.homberger@web.de>
2024-01-08 19:26:03 +00:00
Jon Jensen
00fbfa754c Fix noisy runs-on error logging (#2102)
Move the logging back up a level to fix a minor logging issue introduced in #2088

`RunContext`s for composite actions have dummy/blank `Job`s with no `runs-on`,
meaning their calls to `withGithubEnv` would result in an inaccurate log message
complaining that `'runs-on' key not defined in ...`

Co-authored-by: Jason Song <i@wolfogre.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2023-12-16 23:04:54 +00:00
ChristopherHX
1b10028447 fix: IsHost is defined as false on windows (#2093)
* fix: IsHost is defined as false on windows

* Update docker_run.go

* Update docker_run.go
2023-12-16 14:46:17 -08:00
Jon Jensen
55477899e7 Evaluate if condition when calling a reusable workflow (#2087)
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: ChristopherHX <christopher.homberger@web.de>
2023-11-12 20:01:32 +00:00
raffis
04011b6b78 feat: support runs-on labels and group (#2062)
Signed-off-by: Raffael Sahli <raffael.sahli@doodle.com>
Co-authored-by: ChristopherHX <christopher.homberger@web.de>
2023-11-12 19:46:38 +00:00
Björn Brauer
c8f847d82d Evaluate all service values (#2054)
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2023-11-12 18:30:21 +00:00
ChristopherHX
74b0fe8ba9 fix: (#2075)
network-scoped alias is supported only for containers in user defined networks

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2023-11-12 18:09:25 +00:00
Jon Jensen
18b4714e38 Don't set GITHUB_TOKEN (#2089)
This needs to be explicitly in the `env` to be consistent with GitHub

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2023-11-12 17:52:08 +00:00
Jon Jensen
610358e1c3 Support array expressions in runs-on (#2088)
* Support array expressions in runs-on

* Simplify appproach to use EvaluateYamlNode, fix case-sensitivity bug

---------

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2023-11-12 17:40:06 +00:00
Andreas Taylor
1c16fd1967 Use unique name for reusable workflow (#2015)
Co-authored-by: ChristopherHX <christopher.homberger@web.de>
2023-11-12 17:21:41 +00:00
Jason Song
5a79256ee4 fix: panic (#2071)
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2023-11-04 14:10:53 +00:00
Sam Foo
ceeb6c160c Add support for service containers (#1949)
* Support services (#42)

Removed createSimpleContainerName and AutoRemove flag

Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: Jason Song <i@wolfogre.com>
Reviewed-on: https://gitea.com/gitea/act/pulls/42
Reviewed-by: Jason Song <i@wolfogre.com>
Co-authored-by: Zettat123 <zettat123@gmail.com>
Co-committed-by: Zettat123 <zettat123@gmail.com>

* Support services options (#45)

Reviewed-on: https://gitea.com/gitea/act/pulls/45
Reviewed-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: Zettat123 <zettat123@gmail.com>
Co-committed-by: Zettat123 <zettat123@gmail.com>

* Support intepolation for `env` of `services` (#47)

Reviewed-on: https://gitea.com/gitea/act/pulls/47
Reviewed-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: Zettat123 <zettat123@gmail.com>
Co-committed-by: Zettat123 <zettat123@gmail.com>

* Support services `credentials` (#51)

If a service's image is from a container registry requires authentication, `act_runner` will need `credentials` to pull the image, see [documentation](https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#jobsjob_idservicesservice_idcredentials).
Currently, `act_runner` incorrectly uses the `credentials` of `containers` to pull services' images and the `credentials` of services won't be used, see the related code: 0c1f2edb99/pkg/runner/run_context.go (L228-L269)

Co-authored-by: Jason Song <i@wolfogre.com>
Reviewed-on: https://gitea.com/gitea/act/pulls/51
Reviewed-by: Jason Song <i@wolfogre.com>
Reviewed-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: Zettat123 <zettat123@gmail.com>
Co-committed-by: Zettat123 <zettat123@gmail.com>

* Add ContainerMaxLifetime and ContainerNetworkMode options

from: b9c20dcaa4

* Fix container network issue (#56)

Follow: https://gitea.com/gitea/act_runner/pulls/184
Close https://gitea.com/gitea/act_runner/issues/177

- `act` create new networks only if the value of `NeedCreateNetwork` is true, and remove these networks at last. `NeedCreateNetwork` is passed by `act_runner`. 'NeedCreateNetwork' is true only if  `container.network` in the configuration file of the `act_runner` is empty.
- In the `docker create` phase, specify the network to which containers will connect. Because, if not specify , container will connect to `bridge` network which is created automatically by Docker.
  - If the network is user defined network ( the value of `container.network` is empty or `<custom-network>`.  Because, the network created by `act` is also user defined network.), will also specify alias by `--network-alias`. The alias of service is `<service-id>`. So we can be access service container by `<service-id>:<port>` in the steps of job.
- Won't try to `docker network connect ` network after `docker start` any more.
  - Because on the one hand,  `docker network connect` applies only to user defined networks, if try to `docker network connect host <container-name>` will return error.
  - On the other hand, we just specify network in the stage of `docker create`, the same effect can be achieved.
- Won't try to remove containers and networks berfore  the stage of `docker start`, because the name of these containers and netwoks won't be repeat.

Co-authored-by: Jason Song <i@wolfogre.com>
Reviewed-on: https://gitea.com/gitea/act/pulls/56
Reviewed-by: Jason Song <i@wolfogre.com>
Co-authored-by: sillyguodong <gedong_1994@163.com>
Co-committed-by: sillyguodong <gedong_1994@163.com>

* Check volumes (#60)

This PR adds a `ValidVolumes` config. Users can specify the volumes (including bind mounts) that can be mounted to containers by this config.

Options related to volumes:
- [jobs.<job_id>.container.volumes](https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#jobsjob_idcontainervolumes)
- [jobs.<job_id>.services.<service_id>.volumes](https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#jobsjob_idservicesservice_idvolumes)

In addition, volumes specified by `options` will also be checked.

Currently, the following default volumes (see a72822b3f8/pkg/runner/run_context.go (L116-L166)) will be added to `ValidVolumes`:
- `act-toolcache`
- `<container-name>` and `<container-name>-env`
- `/var/run/docker.sock` (We need to add a new configuration to control whether the docker daemon can be mounted)

Co-authored-by: Jason Song <i@wolfogre.com>
Reviewed-on: https://gitea.com/gitea/act/pulls/60
Reviewed-by: Jason Song <i@wolfogre.com>
Co-authored-by: Zettat123 <zettat123@gmail.com>
Co-committed-by: Zettat123 <zettat123@gmail.com>

* Remove ContainerMaxLifetime; fix lint

* Remove unused ValidVolumes

* Remove ConnectToNetwork

* Add docker stubs

* Close docker clients to prevent file descriptor leaks

* Fix the error when removing network in self-hosted mode (#69)

Fixes https://gitea.com/gitea/act_runner/issues/255

Reviewed-on: https://gitea.com/gitea/act/pulls/69
Co-authored-by: Zettat123 <zettat123@gmail.com>
Co-committed-by: Zettat123 <zettat123@gmail.com>

* Move service container and network cleanup to rc.cleanUpJobContainer

* Add --network flag; default to host if not using service containers or set explicitly

* Correctly close executor to prevent fd leak

* Revert to tail instead of full path

* fix network duplication

* backport networkingConfig for aliaes

* don't hardcode netMode host

* Convert services test to table driven tests

* Add failing tests for services

* Expose service container ports onto the host

* Set container network mode in artifacts server test to host mode

* Log container network mode when creating/starting a container

* fix: Correctly handle ContainerNetworkMode

* fix: missing service container network

* Always remove service containers

Although we usually keep containers running if the workflow errored
(unless `--rm` is given) in order to facilitate debugging and we have
a flag (`--reuse`) to always keep containers running in order to speed
up repeated `act` invocations, I believe that these should only apply
to job containers and not service containers, because changing the
network settings on a service container requires re-creating it anyway.

* Remove networks only if no active endpoints exist

* Ensure job containers are stopped before starting a new job

* fix: go build -tags WITHOUT_DOCKER

---------

Co-authored-by: Zettat123 <zettat123@gmail.com>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: Jason Song <i@wolfogre.com>
Co-authored-by: sillyguodong <gedong_1994@163.com>
Co-authored-by: ChristopherHX <christopher.homberger@web.de>
Co-authored-by: ZauberNerd <zaubernerd@zaubernerd.de>
2023-10-19 09:24:52 +00:00