Skip to content

Conversation

@SwuduSusuwu
Copy link
Owner

@SwuduSusuwu SwuduSusuwu commented Jun 15, 2025

This branch just exists to backup/store local changes and request reviews.
This is a "draft" pull request, which just exists to show autonomous code review results and encourage crowd sourced reviews (so that all code git push'd to trunk branch, is fit to use).

What to review:

experimental is a mirror for when this branch is in the middle of a partial merge (which, for GitHub pull requests, requires to roll this branch back to the last commit which will merge).
  - The effect is that (while in the process of merges), just the experimental branch has the most new version of code.

@SwuduSusuwu SwuduSusuwu added improve New feature or request help wanted Extra attention is needed good first issue Good for newcomers question Further information is requested todo unimplemented (but advertised) part of the tool labels Jun 15, 2025
@SwuduSusuwu SwuduSusuwu force-pushed the preview branch 4 times, most recently from e9ef643 to 294e114 Compare June 15, 2025 21:02
@SwuduSusuwu SwuduSusuwu force-pushed the preview branch 15 times, most recently from f52109a to 3f37301 Compare June 18, 2025 17:47
SwuduSusuwu added a commit that referenced this pull request Jun 19, 2025
TODO: squash this into the numerous commit which introduce `TensorFlowCns`.
	Those commits are all in the middle of [this active pull request (which has numerous rebases to do)](#67).
Notice: this commiit will not include all individual updates in "lock-step" (which is what this project says to do, but is not practical in active rebases on the `preview` branch).
SwuduSusuwu and others added 28 commits November 2, 2025 12:08
@`ClassCnsTestsLinearArgus`: +`coefficient`, +`bias`; to produce numerous functions.
Is followup to: commit HEAD~1 (@`cxx/ClassCns.hxx`: +`classCnsTests<>()`). TODO: `squash`?
@`posts/VirusAnalysis.md`: include this.
Is followup to: commit HEAD~1 (@`cxx/ClassCns.hxx:classCnsTests<>()`: general use)`), which instruments more forms of tests. TODO: `squash`?
@`posts/VirusAnalysis.md`: include this.
	+`class TensorFlowCns : public Cns`; implements `class Cns : public Object`.
	+`initSynapse()`; helper function for `pseudoRandomSynapses()`.
	+`initScopeRootBack()`, +`pseudoRandomSynapses()`; helper functions for `TensorFlowCns::setupSynapses()`. Is followup to HEAD~10 (+`cxx/ClassCns.hxx:pseudoRandomSynapses()`, helper), which introduced `Cns::pseudoRandomSynapses`.
	+`setupSynapsesImpl()`; helper function for `TensorFlowCns::setupSynapses()`. Is followup to HEAD~5 (+`cxx/ClassCns.hxx:setupSynapsesPostProcess()`), which introduced `Cns::setupSynapsesPostProcess()`.
	+`initScopeRootForward()`; helper function for `TensorFLowCns:process*()`.
	+`printGraphNodes()`; helper function for stacktraces.

[_Cisco-Talos_ asked for a pull request which includes our analysis code](https://github.com/SwuduSusuwu/SusuLib/issues//1) (code which is available though dual licenses; [Apache 2](https://github.com/SwuduSusuwu/SusuLib/blob/trunk/LICENSE) or [GPLv2](https://github.com/SwuduSusuwu/SusuLib/blob/trunk/LICENSE_GPLv2)), but this code relies on an [experimental backend for _TensorFlow_](https://github.com/SwuduSusuwu/SusuLib/blob/preview/cxx/ClassTensorFlowCns.hxx).

@`./compile_commands.json`; include `ClassTensorFlowCns` (required for `clang-tidy cxx/ClassTensorFlowCns.hxx`)
@`./posts/VirusAnalysis.md`; include all this.
@`./README.md#purposes`, @`./cxx/README.md#purposes`; document `./cxx/ClassTensorFlowCns.hxx`.
TODO: through [_Termux_ (smartphone) must have](https://github.com/termux/termux-packages/issues//24657) [`libtensorflow`](https://github.com/tensorflow/tensorflow/tree/master/tensorflow/tools/android) for `SUSUWU_BUILD_OBJECTS()`, `SUSUWU_BUILD_EXECUTABLE()`, `susuwuUnitTests()` with `-DSUSUWU_USE_TENSORFLOW=true`.

+`SUSUWU_CNS_LOCAL_COEFFICIENTS`; local `coefficients`
+`SUSUWU_TENSORFLOW_HAS_DATATYPETOENUM`; use `tensorflow::DataTypeToEnum`.
@`./README.md#optionssetup`; document `-DSUSUWU_CNS_LOCAL_COEFFICIENTS`, `-DSUSUWU_TENSORFLOW_HAS_DATATYPETOENUM`.
@`./cxx/ClassTensorFlowCns.hxx`:
	`TensorFlowCns::{coefficients,biases}`; wrap with `#if SUSUWU_CNS_LOCAL_COEFFICIENTS`.
//	`TensorFlowCns::dumpTo()`; `#if !SUSUWU_CNS_LOCAL_COEFFICIENTS`, use local stack to process `root.WithOpName("coefficients")` and `root.WithOpName("biases")`
@`cxx/AssistantCns.*xx`, @`cxx/VirusAnalysis.*xx`: if `SUSUWU_USE_TENSORFLOW`, use `class TensorFlowCns`.
@`cxx/ClassTensorFlowCns.hxx`: `#ifdef SUSUWU_USE_TENSORFLOW` wraps whole file, fixes "tensorflow/cc/framework/ops.h not found".
@`posts/VirusAnalysis.md`: include this.
Is followup to: commit HEAD~1 (+`cxx/ClassTensorFlowCns.hxx`; `Cns` on TensorFlow)
TODO?: @`cxx/Macros.hxx`: +`SUSUWU_USE_TENSORFLOW` (`#include` comments list this, but for now definition is just from `./build.sh`)
	on inputs (`outputNorms()` on outputs) for `setupSynapses()` and `processTo*()`.
	Purpose: solves issues which some optimizers and activation functions have with large values.

@`posts/VirusAnalysis.md`; include this.
Is followup to: commit HEAD~3 (+`cxx/ClassTensorFlowCns.hxx`; `Cns` on TensorFlow), which introduced `class TensorFlowCns`.
	TODO: `squash` into commit HEAD~3 --- or into commit HEAD~4 (+`ClassCns::dumpTo()`, +`ClassCns::loadFrom()`), if that commit is moved after what is now HEAD~3.
Unused. Current architecture just implements SLP (single-layer-perceptron).
TODO: Those macros serve as reminders to [choose how to implement `layersOfNeurons`](https://github.com/copilot/share/427f408e-08e0-8c80-9151-f24920212857).

@`posts/VirusAnalysis.md`; include all this.
@`cxx/ClassTensorFlowCns.hxx`:
	@`setupSynapses()`; store `root.WithOpName("biases")` (membrane potentials) into `TensorFlowCns.biases`.
	@`processTo*()`; add `root.WithOpName("biases")` to output.
	+`getParameterCount()`; include count of biases into total parameter count.

Is followup to commit HEAD~1 (+`cxx/ClassTensorFlowCns.hxx`; `Cns` on TensorFlow)`; loss), which stores connectome coefficients (also known as "synaptic strengths") into `TensorFlowCns.coefficients`. TODO: squash this?
@`posts/VirusAnalysis.md`; include all this.
	allows to disable `biases` (matrix add), for formulas which just use `MatMul`.

@`cxx/ClassTensorFlowCns.hxx`: wrap all definitions (and uses) of `biases` with `SUSUWU_CNS_IF_BIAS()`.
Is followup to: commit HEAD~1 (+`cxx/ClassTensorFlowCns.hxx:biases` (potentials)). TODO: `squash` this?

@`posts/VirusAnalysis.md`: include all this.
	Is followup to: commit b10cabe (+`sh/Includes.sh`, @`sh/README.md#purposes`)
	Use `SUSUWU_DEPENDENCY_INCLUDE()` to search for `libtensorflow` include paths. If found, put into `FLAGS_USER` (which goes into `CXXFLAGS`). If not found, suggest how to install `libtensorflow`.
		Is followup to: commit b7c33e0 (+`sh/make.sh:SUSUWU_DEPENDENCY_INCLUDE()`).

	If `SUSUWU_SETUP_BUILD_FLAGS_CONDITIONAL "" "-DUSE_TENSORFLOW_CNS" "-ltensorflow" cxx/ClassTensorFlowCns.hxx` is success, store. If not, restore backup flags plus suggest how to troubleshoot.
		Is followup to: commit HEAD~5 (@`README.md#optionssetup`; +`USE_TENSORFLOW_CNS`), which introduced `USE_TENSORFLOW_CNS`. TODO: `squash` this?

	+`SUSUWU_INCLUDES_LIBTENSORFLOW_ERROR`, +`SUSUWU_INCLUDES_LIBTENSORFLOW_PASS`; prevents re-execution of `${CXX} ${CXXFLAGS} ${SUSUWU_INCLUDES_LIBTENSORFLOW_FLAGS} -c "${SUSUWU_INCLUDES_LIBTENSORFLOW_TEST_PATH}"` from `SUSUWU_TEST_BASH()`.

@`build.sh`: use `SUSUWU_INCLUDES_LIBTENSORFLOW()`.
@`README.md#purposes`, @`sh/README.md#purposes`: document `SUSUWU_INCLUDES_LIBTENSORFLOW()`.

Improves <https://github.com/SwuduSusuwu/SusuLib/issues//6>
@`build.sh`: +`SUSUWU_IS_VIRTUAL`, +`SUSUWU_INSTALL_TENSORFLOW`; installs prerequisites + TensorFlow. TODO: move those into `sh/Includes.sh`?
@`README.md#optionssetup`; document this.
@`.gitignore`, @`sh/make.sh:SUSUWU_BUILD_CTAGS()`; exclude `./tensorflow/`, `./ml_dtypes/`.
@`sh/Includes.sh:SUSUWU_INCLUDES_LIBTENSORFLOW()`: use `SUSUWU_INSTALL_TENSORFLOW`
Is followup to: commit HEAD~1 (+`sh/Includes.sh:SUSUWU_INCLUDES_LIBTENSORFLOW()`), which gives TensorFlow paths to `${CXXFLAGS}` (if found).
@`sh/Includes.sh:SUSUWU_INCLUDES_LIBTENSORFLOW()`:
	+`export TF_CPP_MIN_LOG_LEVEL=0`; so TensorFlow debug messages are shown.
	If not `${USE_GOOGLEAPIS_TENSORFLOW}`, prefer `libtensorflow_cc` C++ package (which has system headers to use).
	`libeigen3-dev` search improved.
	Prefer system `libeigen3-dev`; fixes "tensorflow/core/framework/tensor.h:26:10: fatal error: /framework/tensor.h:26:10: fatal error:  unsupported/Eigen/CXX11/Tensor: No such file or directory" on Ubuntu.
	Is followup to: commit HEAD~2 (+`sh/Includes.sh:SUSUWU_INCLUDES_LIBTENSORFLOW()`). TODO: `squash`?

@`build.sh`:
	@`SUSUWU_IS_VIRTUAL`: improved.
	@`SUSUWU_INSTALL_TENSORFLOW`: moved to `sh/Includes.sh:SUSUWU_INCLUDES_LIBTENSORFLOW()`.
	Is followup to: commit HEAD~1 (+`SUSUWU_IS_VIRTUAL`, +`SUSUWU_INSTALL_TENSORFLOW`). TODO: `squash`?
Notice: not tested (Termux nor Ubuntu's package managers have this).
	+`SUSUWU_INCLUDES_LIB_TENSORFLOW_HAS_PROTOBUF`: is set if `.pb.h` source (which `bazel` produces) found.
	If `!SUSUWU_INCLUDES_LIB_TENSORFLOW_HAS_PROTOBUF`, use `protoc` to convert `.proto` sources into those.
	TODO: `squash` this?
Fixes "absl/status/status.h: No such file or directory" on Ubuntu.
Uses `-ltensorflow_cc` instead of `-ltensorflow`.
Fixes `ld.lld: error: undefined symbol` errors.
TODO: scan for include paths to use. Improve codeflow.
Is followup to: commit HEAD~6 (+`sh/Includes.sh:SUSUWU_INCLUDES_LIBTENSORFLOW()`). TODO: `squash`?
@`build.sh`; if in GitHub workflow, `SUSUWU_ABORT_ON_FIRST_ERROR=true`.
@`sh/Includes.sh:SUSUWU_INCLUDES_LIBTENSORFLOW`; use `SUSUWU_ABORT_ON_FIRST_ERROR`.
TODO: `squash`?
+`./cxx/ClassTensorFlowCns.hxx:SigmoidCrossEntropyWithLogits()`; `tensorflow::ops::SigmoidCrossEntropyWithLogits()` was not found.
@`./cxx/ClassTensorFlowCns.hxx:setupSynapses()`;
	For binary classification loop, use `SigmoidCrossEntropyWithLogits()`.
	For linear regression loop, use `tensorflow::ops::SquaredDifference()` (`tensorflow::ops::MeanSquaredError()` was not found)

Is followup to HEAD~5 (+`cxx/ClassTensorFlowCns.hxx`; `Cns` on TensorFlow). TODO: `squash`?
TODO: include arguments, which choose regression formulas.
TODO: replace checks of `outputMode` with +`Cns::isClassification()`, +`Cns::isRegression()`.

@`posts/VirusAnalysis.md`; include all this
	@`./cxx/ClassTensorFlowCns.hxx`: +`TensorFlowCns::dumpTo()`, `TensorFlowCns::loadFrom()`

@`./posts/VirusAnalysis.md#source-code`; include all this.
	validation holdout values (TODO: use those for early stops).
	+`SUSUWU_CLASSCNS_SHUFFLE`; shuffle inputs to `setupSynapses()` so validation set is random.
	@`README.md#optionssetup`; document `-DSUSUWU_CLASSCNS_SHUFFLE`.
	Is followup to: commit HEAD~16 (+`cxx/ClassTensorFlowCns.hxx`; `Cns` on TensorFlow), which introduced `TensorFlowCns::setupSynapses()`. TODO: squash this?

@`posts/VirusAnalysis.md`; include all this.
@`cxx/ClassTensorFlowCns.hxx:setupSynapses()`; return if `loss < desiredLossThreshold`.
Is followup to commit HEAD~2 (@`class TensorFlowCns::setupSynapses()`; loss), which introduced `desiredLossThreshold`.
@`posts/VirusAnalysis.md`; include this.
@`cxx/ClassTensorFlowCns.hxx:setupSynapses()`; if `loss` improves less than `Cns.minLossDelta` for `Cns.patience` loops, abort training loop.
Is followup to commit HEAD~3 (@`class TensorFlowCns::setupSynapses()`; loss), which introduced `desiredLossThreshold`.
@`posts/VirusAnalysis.md`; include all this.
@`cxx/ClassTensorFlowCns.*xx`;
	+`classTensorFlowCnsTests()`, +`classTensorFlowCnsTestsNoexcept()`
	Is followup to: commit HEAD~2 (@`cxx/ClassCns.hxx`: +`classCnsTests<>()`), which introduces the template those use.

@`cxx/main.hxx:SusuwuUnitTestsBitmask`; +`susuwuUnitTestsTensorFlowCnsBit`
@`cxx/main.cxx`; use `classTensorFlowCnsTestsNoexcept()`
@`sh/Includes.sh:SUSUWU_INCLUDES_LIBTENSORFLOW()`; try to compile `cxx/ClassTensorFlowCns.cxx` to determine whether or not to set `-DSUSUWU_USE_TENSORFLOW`.
@`README.md#purposes`, @`cxx/README.md#purposes`; document `class TensorFlowCns`, `classTensorFlowCnsTests()`, `classTensorFlowCnsTestsNoexcept()`.
@`posts/VirusAnalysis.md#source-code`; include all this.
Purpose: diagnosis of `[2025-06-12 06:51:15] [build-stdout] [./cxx/ClassSys.hxx:93: Error: classTensorFlowCnsTests() {throw std::exception("classTensorFlowCnsTests(); label[2000] - output[2000] == 4002.000000, but float epsilon is just 0.000200.");}]`
TODO: `squash` this?
@`posts/VirusAnalysis.md`; include this.
This reduces the first of `cxx/ClassCns.hxx:classCnsTests()`'s `processToInt()` test's `worstLoss` from 2242 to 251, the second from almost 2552 to 564.
TODO: figure out what was wrong with the original codeflow.

@`classCnsTests()`; reduce `epsilon` arguments to new `worstLoss` values.
@`posts/VirusAnalysis.md`; include this.
Is followup to: commit HEAD~1 (@`posts/CnsCompress.md`; English+Markdown improved), which is set to `git push` into `trunk` branch (but this is not, until `class TensorFlowCns` is).
	This is just to ensure source code is still compatible with C++20 projects. `./build.sh`'s minimum requirement is still C++11, not C++20.
TODO: have [_GitHub_'s scan tools](https://github.com/SwuduSusuwu/SusuLib/actions/runs/13170590727/job/36760093264) build with minimum + maximum compatible versions of C++.

@`sh/Includes.sh:SUSUWU_INCLUDES_LIBTENSORFLOW_FLAGS`: ditto (`libtensorflow`'s minimum requirement is still C++17, not C++20).
@`./posts/VirusAnalysis.md`: include this.
@`./cxx/ResultList.hxx`:
	@`struct ResultList`: is now `typedef<class Set> struct ResultListBase`.
	+`typedef struct ResultListBase<std::unordered_set<ResultListHash>> ResultList; /* `unordered_set` is the O(1) formula. */
	+`typedef struct ResultListBase<std::set<ResultListHash>> ResultListSorted; /* `set` is an O(log n) formula, but is deterministic (sorted). */`
+`./posts/JavaVersion.md#table-of-contents`: will include more source in future.
Is followup to: commit 3e99514 (+`posts/JavaVersion.md`: of `cxx/ClassObject.hxx`). TODO: `squash` this
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

good first issue Good for newcomers help wanted Extra attention is needed improve New feature or request question Further information is requested todo unimplemented (but advertised) part of the tool

Projects

None yet

2 participants