It looks like the github windows build environment is not correctly
inheriting environment variables. Which is strange, as that has not
been changed with the recent updates. Include the vars directly as a
workaround (and plan to refactor later)
It is difficult to maintain one build system, let alone two - especially
if one build system is known to be better maintained and the other build
system continues to be used - with bug and feature requests raised
against it.
The maintainers are aware that there are people using the CMake system,
but believe that the extra maintainability of only having one build
system, combined with the knowledge that the current CI system
demonstrates that all supported architectures are working with the
Makefiles make the use of these Makefiles a more sustainable direction.
For a moment there, I was starting to think that I might have found
a good feature in cmake, but then it turned out that it had specific
ordering requirements and strange cryptic functions that error out
in mysterious ways.
If I am going to have an antique build engine with special quirks that
I must learn in order to use it, I may as well choose the one that is
not also trying to be clever and hiding its internal magic from me.
Which means that I still prefer Makefiles - they are more debuggable.
* Ensure all options start off for cmake too
* Attempt to make the build time config logic in the cmake process match the makefile process
* Add a simple muscle memory helper for cmake
* Ask them what they are smoking, then tell them to put it out
* Avoid occasional stale package cache errors
* First attempt at a openwrt CI
* Fix action - helps if I dont forget the syntax half way through writing it
* Try building /this/ branch for the openwrt CI
* Try a build with openssl configured
* Dont enable the n2n package until we have built the environment - makes errors easier to see and clearly related to n2n
* Attempt to speed up the openwrt build
* Upload any created ipkg packages
* Dont test with openssl for the moment
* Attempt to speed up openwrt build using a cache of the build dir
* The make defconfig run turns all the built binaries stale, so stop caching them. Also use a real ref for the cache key
* Minor text name changes
* Address yamllint concerns
* Attempt to simplify and document missing parts of the openwrt makefile
* Attempt to fix mystery openwrt make error
* Rename build job name
* Avoid nested checkouts, use two separate dirs for the two checkouts in this build
* Move the n2n checkout to earler, allowing us to skip one defconfig run
* We are going to need working tags from the n2n repo, so ensure we unbreak the github checkout braindamage
* Calculate and save the n2n version string
* Prepare the way to pass the correct external vars into the openwrt build
* Hook calculated build variables into the openwrt package definition
* Update artifacts source to match moved checkout dir
* Pass env vars in to the make
* Allow version script to be influenced by external vars
* It will help if I use the same variable names everywhere
* Add more version variable calculation overrides
* Configure openwrt to use the external git checkout instead of their create-tar-then-extract dance
* Using the correct syntax for ifdef will help significantly
* Use as many jobs as we have cpus
* As the USE_SOURCE_DIR option allows us to use a full git checkout, we do not need to hack the version.sh to allow overrides
* Ensure scripts/version.sh works from anywhere
* Remove unneeded variables
* Update openwrt build documentation to match the new build process
* Catch failure to cd as per shellcheck suggestion
* Limit lengthy openwrt builds to manual triggers or on a release
* Also run on specially named branches
* Break list into separate lines for easier future editing
* Make it clear which hash we are testing
* Coverage for all pearson hash variants
* Initial coverage for auth.c
* Add some more auth.c coverage
* Ensure new bins are added to both build systems
* Ensure new source files are fully linted
We aim to always build with the full version number from the git
checkout, however if the current build is not being made from a git
repository, we need to fallback to using the version number from
the "VERSION" file.
This patch changes the choice of the fallback path from just assuming
that any git repo is our git repo to explicitly looking for a `.git` dir
in the top of our checkout.
Some integrated build systems end up doing a tar extract of the n2n
source code inside their build-system git repo, and the old fallback
check was failing in that case.
We still need to identify these build systems and work out a way to
provide them with the full and correct version number for their build,
but this patch should make things more reliable until then.
* Address shellcheck concerns with n2n_gateway.sh script
* Rename n2n_gateway.sh to live with all the other scripts (and update docs)
* Rename hack_fakeautoconf to make it clearly a shell script
* Address shellcheck concerns with hack_fakeautoconf.sh
* Rename python scripts to match other n2n tools
* Fix windows compile - when streamlining the use of hack_fakeautoconf.sh, I forgot to update all uses of this script
* Factor build packages out into a more maintainable list
* Create a location for scripts to live
* Provide a make target to return the source dir as close as reasonable to the original distributed state
* Add a code lint step, checking the coding style
* Change test harness as recommended by shellcheck
* Ensure we actually have the linter tool installed
* Use the correct directory for cmake to run the tests
* Adjust for the older uncrustify in the current github ubuntu-latest
* Make one file pass the linter
* Integrate the lint with the existing test workflow
* Add files with minimal changes needed to the linter
* Add more files with minimal changes needed to the linter
* Dont build binaries if we fail the lint test
* Update the phony targets with the lint steps
* Ensure the flake8 package is installed in the new lint workflow job
* Use the makefile to drive the packages needed to install for linting
* No need to add dependancies on lint, just rely on the workflow status to show failure
* Update the scripts dir README to reflect current assumptions
* Rename and briefly document the indent.sh script
* Fix the ignore to ignore the right Makefile
* Rename the test_harness script to make it clear it is a shell script
* Provide a master lint make target and add a shell script lint tool
* Elminate stray tabs
* Drop include/auth.h from linter - there are inconsistant results with function definitions when using the current uncrustify rules
* Ensure that recent code additions pass the linter
* Include some of the more obviously correct lint fixes to edge_utils.c
* Refactor edge JSON api into its own source file
* Use shorter names for static management functions
* Implement a JSON RPC way of managing the verbosity
* Tidy up help display in n2nctl script
* Make note of issue with implementing the stop command
* Implement a JSON RPC call to fetch current community
* Make n2nhttpd time value be more self-contained
* Make n2nhttpd order more closely match the existing management stats output
* Wire up status page to the verbosity setting
* Add JSON versions of the remainder of the edge management stats
* Add new file to cmake
* Properly define management handler
* Only update the last updated timestamp after a successful data fetch
* Function and types definition cleanup
* Force correct type for python scripts mgmt port
* Implement initial JSON API for supernode
* Fix whitespace error
* Use helper function for rendering peers ip4 address
* Proxy the auth requirement back out to the http client, allowing normal http auth to be used
* Ensure that we do not leak the federation community
* Use the same rpc method name and output for both edge and supernode for peers/edges
* Allow n2nctl to show raw data returned without resorting to tricks
* Make n2nctl pretty printer understandable with an empty table
* Use the full name for supernodes RPC call
* Use same RPC method name (but some missing fields) for getting communities from both edge and supernode
* Add *_sup_broadcast stats to edge packet stats output
* Refacter the stats into a packetstats method for supernode RPC
* Even if I am not going to prettyprint the timestamps, at least make all the timestamps on the page the same unit
* Simplify the RPC handlers by flagging some as writable and checking that in the multiplexer
* Remove invalid edges data
* Avoid crash on bad data to verbose RPC
* Avoid showing bad or inconsistant protocol data in communities RPC
* Minor clarification on when --write is handled
* Make linter happy
* Fix changed method name in n2nhttpd
* Move mainloop stop flag into the n2n_edge_t structure, allowing access from management commands
* Implement edge RPC stop command
* Move mainloop stop flag into the n2n_sn_t structure, allowing access from management commands
* Implement supernode RPC stop command
* Allow multiple pages to be served from mini httpd
* Extract common script functions into a separate URL
* Handle an edge case in the python rpc class
With a proper tag-based demultiplexer, this case should be a nop,
but we are single-threaded and rely on the packet ordering in this
library.
* Add n2nhttpd support to query supernode using urls prefixed with /supernode/
* Handle missing values in javascript table print
* Add another less filtering javascript key/value renderer
* Add a supernode.html page to the n2nhttpd
* Address lint issue
* Mention the second html page on the Scripts doc
* Remove purgable column from supernode edges list - it looks like it is rarely going to be set
* Add a simple one-line example command at the top of the API documentation
* Acknowledge that this is not the most efficient protocol, but point out that it was not supposed to be
* Make it clear that the n2nctl script works for both edge and supernode
* Fight with inconsistant github runner results
* Turn off the /right/ coverage generator
* Reimplement JSON mgmt with clear separation of read/write actions
* Reduce boilerplate by using a table driven command definition for json mgmt commands
* Port tools to use new json api
* Add a basic authentication for json mgmt commands
* If a auth key is given, it must match
* Add auth key to management scripts
* Add a flag bitfield to clearly turn the tag param into a options list
* Allow simple pass-through of any command from n2nctl
* Convert the n2nctl to use an object oriented interface
* Handle sigpipe in the n2nhttpd - this happens if the remote client disconnects unexpectely
* Remove some repetition from the server
* Use the correct options to allow reuseaddr
* Dont generate a scary message on ctrl-c
* Convert n2nhttpd to use object based RPC
* Use the same longopt for both tools
* Pass any extra args through to the RPC
* Add some documentation for the scripts in the repository
* Spelling fix
* Add documentation for the JSON reply mangement API
* Add management commands to show data in JSON format
* Add a script to query the JSON management interface
* Suprisingly, the github runner does not have flake8 installed
* Add n2nctl debugging output to show the raw data received from the JSON
* Ensure well known tag wrap-around semantics
* Try to ensure we check every edge case in the protocol handling - only valid packets are allowed
* Add a very simple http to management port gateway
* Fix the lint issue
* Provide a minimal reimplementation of our autoconf, to try windows builds
* Try building with windows
* Fix thinko in spelling
* Ensure shell script runs inside a shell
* Add a hack to aid include discovery
* Just keep adding tech debt...
* Assume that we will have slashes in some of the replacement strings and avoid that char with sed
* Restore one slash
* Hack around the tools makefile interdependancy bug
* A correct cflags include hack for each compile dir
* Ensure we link against winsock (note, even though this says 32bit, it should link the 64bit library ... I think)
* Bad link ordering if we dont use LDLIBS
* Remove unused make variable
* Remove makefile duplication using inheritance (this does mean you can no longer cd tools; make, but must do make tools)
* Add missing library for win32
* Show OS variable
* Make hack autoconf more robust for tests on non gitlab runners
* Remove no longer used substitutions from hack autoconf
* Add missing include path to tools under win32
* Build the win32 subdir when the compiler is Msys
* The different subdirs have different dependancies
* Ensure we can find the include files
* Fix library link ordering
* Ensure the tools dir can find the special win32 lib
* Deal with the differing basic type sizes on both linux/64bit and windows/64bit
* Document the steps to mimic the github windows/mingw build locally - to allow for simpler debugging
* Ensure branch name in instructions matches my test branch name
* Clarify the shell needed to build with mingw
* Since the makefile depends on knowing the OS, raise a fatal error if we cannot determine this
* Handling different compile environments is hard.
- Linux: sane and reasonable results for both uname -s (=Linux) and
uname -o (=GNU/Linux)
- Windows/Mingw: insane results for uname -s
(=MSYS_NT-$MAJOR.$MINOR-$BUILDNR) but sane results for uname -o (Msys)
- Macos: sane results for uname -s (=Darwin) but does not support
uname -o at all
* Revamp the way that Mingw is detected
* Avoid attempting to generate gcovr report when running under windows
* Whoops, isolate the right step
* Fix spelling mistake
* win32/Makefile: Remove unused setting and add comment
* ensure that all win32 includes use the same expected path
* Allow simpler cross compilation by letting configure pass the CC and AR environment through
* Avoid multiple '_CRT_SECURE_NO_WARNINGS redefined' warnings
* Convert to a consolidated CONFIG_TARGET variable to select any different compile options
* Use the more generic printf defines to avoid warnings on mingw
* Update mingw build docs
* English better for reader happy make
* Address a number of mingw compiler warnings
* Fix Visual C compile
* Be sure to document some of the hacky nature of the mingw build
Hamish Coleman
Tony
Luca Deri
John Niang
Zoltán Lajos Kis