It was our fault all the time, but we didn’t know how and why and we wanted to know. We wanted to know why the build of our Angular 6 application sometimes worked and sometimes didn’t when we used the same Node+Npm versions every time. I suspected the environment differences, PATH perhaps, but the punch line was… well, punch line really.
How did it manifest?
When I ran ng serve or ng build on the command line (Git bash or cmd when I was desperate), I got this error:
Progress Plugin Invalid Options options should NOT have additional properties options should pass "instanceof" keyword validation options should match exactly one schema in oneOf ValidationError: Progress Plugin Invalid Options options should NOT have additional properties options should pass "instanceof" keyword validation options should match exactly one schema in oneOf at validateOptions (...\node_modules\webpack\node_modules\schema-utils\src\validateOptions.js:32:11) at new ProgressPlugin (...\ui\node_modules\webpack\lib\ProgressPlugin.js:90:3) ...
Now the funny thing is that it worked fine in IntelliJ IDEA 2018 or when ran from Gradle using Moowork’s Node plugin.
Even funnier, it worked fine for a guy on our team who used his system default Node 8.x, while we were all using Node 10.15.1 with Npm 6.8.0. The same version IDEA used, as we instructed it to, and the same version Gradle Node Plugin used as well.
How to run specific Node with specific Npm?
In order to make our builds reliable we wanted to use the same version of Node and Npm for some version of the project every time – even in the future when we build some older commit. We want to “lock” the versions of Node and Npm, so to say, whether there is some other Node installed (e.g. globally) or none at all.
For this reason we have additional shell scripts in the project that do this for us, so instead of just running npm (a random one), we run ./npm.sh from our UI application directory. Alternatively we can source setenv.sh included in the project that prepares the environment (current shell). Originally I didn’t want to alter the PATH variable as well, but in the end – specifically for Node and Npm – I did so. Our setenv.sh prints something reassuring like:
NODE_HOME: /c/work/tools/nodejs/node-v10.15.1-win-x64 NPM_HOME: /c/work/tools/npm/npm-v6.8.0
With this you can run known versions of Node like:
$NODE_HOME/node -v v10.15.1
Now Node installation has Npm too, but not the one I wanted:
$NODE_HOME/npm -v 6.4.1
To run what I want you have to do kinda obscene:
$NODE_HOME/node $NPM_HOME/node_modules/npm/bin/npm-cli.js -v 6.8.0
That’s better.
Obviously, if we simply add both homes to the PATH – to its start, to take precedence before other possible default node/npm – than it all works without a hassle:
PATH=$NPM_HOME:$NODE_HOME:$PATH npm -v 6.8.0
It’s important to put $NPM_HOME first, otherwise npm from $NODE_HOME is used.
What about Angular CLI?
Now let’s go back to the state where Node/Npm is not on the path, let’s try Npx to run Angular CLI. So we have NPM_HOME set, let’s try to ignore NODE_HOME first:
$NPM_HOME/npx ng version ... Angular CLI: 6.2.9 Node: 10.15.3 ... Angular: 6.1.10
As I was in our UI application (directory containing node_modules), I got the right version of Angular and Angular CLI, but not the expected version of Node – it’s not pinned to 10.15.1, but a bit newer found on PATH. We really have to use the long incantation:
$NODE_HOME/node $NPM_HOME/node_modules/npm/bin/npx-cli.js ng version
When I do the same in non-project directory, output changes (command is the same):
Angular CLI: 1.6.6 Node: 10.15.1 ... Angular: ...
The Node version is right, but CLI is something I installed ages ago, found on PATH – and there is no Angular installed, as we’re not in any project.
So – given the current working directory is Angular project directory we don’t need PATH modified. And we can reproduce the error from the beginning of the article:
$NODE_HOME/node $NPM_HOME/node_modules/npm/bin/npx-cli.js ng serve Progress Plugin Invalid Options options should NOT have additional properties ...etc.
However, it is convenient to have the PATH modified, and aside Node and Npm, I’d add also project’s node_module/.bin directory, preferably as absolute path. I’m not talking about system settings, I’m talking about tweak done by project’s setenv.sh script.
. path-to/setenv.sh … echo $PATH | tr ':' '\n' | head -3 /c/path-to-our-project/app/ui/node_modules/.bin /c/work/tools/npm/npm-v6.8.0 /c/work/tools/nodejs/node-v10.15.1-win-x64
That looks good. Now I can run the right ng directly and it will use the right node and npm. But there is still our problem with Progress plugin that does not occur on CI server.
And does not occur in IDEA. Except it does after we upgraded to IDEA 2019. Now we knew all the time that we want to fix it, but I was restless – I didn’t want to just get rid of it, I wanted to know why it works sometimes! I simply want to understand.
The day I was introduced to Process Hacker
So we decided to compare the runtime environment of both processes – the one working OK and the one failing. Task manager can get you only that far, colleague of mine found Process Hacker and I can only recommend it.
Working process was easy to watch as it took its time to build the application. The failing process was much faster, but we managed to get its environment as well (Enter on the process to open the Properties window and there the Environment tab).
There was no obvious difference only some minor ones we could easily refuse as the cause of malfunction. Both processes were the same Node executables, everything seems to be OK.
And the difference was…
We decided to exploit the fact we’re working with JavaScript modules and located the file with the “Progress plugin” string – it was node_modules/webpack/lib/ProgressPlugin.js. First we tried to find out what the invalid options are with:
options = options || {};
console.log(options); // conspicuous line added by us
validateOptions(schema, options, "Progress Plugin");
This told us:
{ profile: true, colors: true }
And we didn’t find any options like this anywhere in our project files (like in angular.json).
More funny thing was that no output was produced by the scenarios when it worked (IDEA 2018 or CI/Gradle build). Now, sometimes I don’t believe anything, so I put process.exit() there and double confirmed that while it didn’t terminate the working case, it did in case of failing scenario.
So the difference was that in working case it did not use Progress plugin at all. Now why? Here my long term experiences kicked in and I tried:
$NODE_HOME/node $NPM_HOME/node_modules/npm/bin/npx-cli.js ng serve | cat
So it’s one of those previously failing commands simply piped to cat. And it worked. It’s so obvious now. Progress plugin indicates progress in a nice way and to do so requires a terminal capable of moving the cursor.
Otherwise it’s disabled – I’m not sure whether something disables it or the plugin disables itself, but this does not bother me anymore.
The fix
So we find the reason for different behaviour and it was time to find the fix. In the end it was old and/or incompatible version of @angular-devkit/build-angular. It was ~0.8.0 in our package.json, and it either had this bug or interfered with some other versions of other packages. I have to admit that some of our upgrades were a bit mismanaged and I’m fully aware of this, but that’s beyond the scope of the story.
We upgraded to ~0.11.0 and later to ~0.13.7 and it seems to work just fine with the rest of our dependencies on Angular 6.1.
Talking about those tildes… I have to write another story soon.