Background
When creating a PR for model class modification, GitHub will run a code validation on that module. But for the related micro-service modules which related to this module, will not be checked.
Meanwhile, programmer who created this PR, may not know how many services import this module, precisely speaking he will not know which services have imported the open-API library generated by openApiGenerator. So, we cannot expect he could start the code validation for the other services.
Could we start code validation for all relevant services automatically?
Assumption
Depends on GitHub Actions, can we find out all relevant services and pass it to code validation workflow? For achieving that goal, we need some steps to follow.
Write a script to find out how many micro-services are importing libraries of this module
Store that service names as a variable and pass it to another GitHub workflow job
The next workflow job is to read the variable passed, start code validation using a loop one by one or simultaneously.
Step-1 Collect the name of services
After some attempts, we can gather service names which imported this module, and this based on the following fact.
We already know the module name or open api client lib name of this module.
All the services maintaining dependency using gradle, and these dependencies are maintained inside build.gradle.kts
Account-manager for example, will generate an open api client lib with group id “ph.safibank.customermanager” and the project name is “customer-manager-api-client”, so the following script will find out all the services imported this lib.
% find ./services/ -name "build.gradle.kts" -d 2 | \
xargs grep "ph.safibank.customermanager:customer-manager-api-client" | \
awk {'print $1'} | \
sed -e 's/.\/services\/\///g' -e 's/\/build.gradle.kts://g'
infobip-email-gateway
transaction-history-manager
product-manager
audit-log-manager
macrokiosk-email-gateway
backoffice-manager
transaction-processor-manager
infobip-sms-gateway
Step-2 Using outputs of first step and pass it
GitHub can reuse the output of one step or job and pass it to other steps or jobs. we can pass the bash array variable to the output and read it in other jobs.
Also readed document of repository_dispatch event to pass more payload which matrix strategy also supported. But that type needs send payload via http post. That means when step 1 executed, need to converter value of parameter to a json string. Need to consider using bash script if supported or by python. BUT, this method have a big disadvantage, that we cannot trace the result of code validation.
Actually, we can use fromJson() to parse a Json string to json, and assigned it to matrix configuration. This means we need to convert the output from a bash array to json string. For simplication, we do not use python or other language or platform, convert to JsonArray string using for-loop of bash. Segment of yml config should like follow.
jobs:
collect-services:
runs-on: ubuntu-latest
outputs:
services: ${{steps.collect.outputs.services}}
steps:
- name: collect importers
id: collect
run: |
services=`
find ./services/ -name "build.gradle.kts" -maxdepth 2 | \
xargs grep "ph.safibank.customermanager:customer-manager-api-client" | \
awk {'print $1'} | \
sed -e 's/.\/services\///g' -e 's/\/build.gradle.kts://g'
`
# original bash array
echo $services
# convert to json string using for-loop
result="["
for service in $services
do
result+="\"$service\","
done
result+="]"
echo $result
# assign json string to output param
echo "::set-output name=services::$result"
Step-3 Checking related services
Unfortunately, we cannot start loop jobs by iterate variable, the reusable workflows a fixed by yaml.
Considering run code validations of each service inside one job. Disadvantage is the code validation will executed one by one, this will extends the execute time of code validation times that before.
GitHub provides matrix strategy, which can start several jobs simultaneously based on the matrix value we set. On step 2, we generated a Json Array string, and GitHub actions can parse those string by fromJson() method and start multiple job. see https://docs.github.com/en/actions/using-jobs/using-a-matrix-for-your-jobs
Finality
This whole process will start with on job that read all the gradle config file, filter with package name and decide which services need to validate. Then we parse this result from bash array variable to a json array string. Finally we parse this json array string to matrix strategy param, and let GitHub Actions start code validate.
Demo of GitHub: https://github.com/mostaron/RelatedCheck