commit 21b4a62f33552d6a31c85b509556b93ae8e5fca1
parent 4eef91feef28cd7e9da18d20d1dfe70727cc8b31
Author: richardfeynmanrocks <freifeld.david@gmail.com>
Date: Mon, 13 Jul 2020 08:31:08 -0700
Added to README
Diffstat:
| M | readme.md | | | 77 | ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++----------------- |
1 file changed, 60 insertions(+), 17 deletions(-)
diff --git a/readme.md b/readme.md
@@ -1,11 +1,11 @@
## About
-Jacobian is a work-in-progress machine learning library written in C++ designed to run as fast as possible. Parallelization will be achieved by utilizing Google's powerful MapReduce framework with [a custom implementation in C](https://github.com/richardfeynmanrocks/mapreduce), and the compiled nature of C++/C as well as the optimizations enabled by compilers enable further speedups. On the same benchmark task (of running a small neural network for 50 epochs on a specified dataset) In some preliminary benchmarks Jacobian has ran up to ~90-110x faster than a simple Keras program. This library is also easily accessible and initializing and using a neural network can be done in just 10 lines of code.
+Jacobian is a work-in-progress machine learning library written in C++ designed to run as fast as possible while still being simple to use. Jacobian is accessible via Python and enables you to write models that train faster in the same amount of code. As of now, Jacobian supports feedforward neural networks and has partial support for convolutional neural networks. ***Note that as Jacobian is a work-in-progress, the latest commit is and will be largely unstable until convolutional networks and multiclass classification are fully implemented.***
## Benchmark Info
-Batch size of model vs. total runtime for this project and Keras (Keras was slow enough that it moves in steps of 20 and starts at a batch size of 20. In reality, the spike at the beginning is much larger for lower batch sizes but it throws the graph off so much you can't see any detail from MIP):
+Batch size of model vs. total runtime for this project and Keras (Keras was slow enough that it moves in steps of 20 and starts at a batch size of 20. In reality, the spike at the beginning is much larger for lower batch sizes but it throws the graph off so much you can't see any detail from Jacobian):
@@ -13,17 +13,68 @@ Average runtime for batch size of 10 for 10 trials:
-Coming soon: A more detailed rundown of the speed of Jacobian vs popular machine learning libraries for Python (and eventually comparisons to C++ libraries as well) as well as a handy and flexible Python script for creating benchmark graphs on the fly.
+**Coming soon:** A more detailed and current rundown of the speed of Jacobian vs popular machine learning libraries for Python (and eventually comparisons to C++ libraries as well) as well as a handy and flexible Python script for creating benchmark graphs on the fly.
+
## Usage
-This library is relatively fit for some use with python, but as of now using Jacobian inside C++ code requires manual building.
+
+Initializing and training a neural network with Jacobian takes just 8 lines of code!
+```python
+import mrbpnn
+net = mrbpnn.Network("../data_banknote_authentication.txt", 10, 0.01, 0.001, 0.5, 0.75)
+net.add_layer(4, "linear")
+net.add_layer(10, "lecun_tanh")
+net.add_layer(1, "linear")
+net.initialize()
+for i in range(50):
+ net.train()
+```
+
+### Rundown
+What's happening in those seven lines?
+
+First, call the Network constructor (from henceforth all functions will be presented in their C++ form). This is where many of the hyperparameters are defined, and you'll need to pass in the path to your data, the desired batch size, learning rate, bias learning rate (which should usually be smaller), regularization strength, and train-val ratio.
+```c++
+Network(char* path, int batch_sz, float learn_rate, float bias_rate, float l, float ratio);
+```
+
+Then add your layers one by one, similar to Keras. You'll need to specify how many neurons there are in the layer and the layer's activation.
+```c++
+void add_layer(int nodes, char* activation);
+```
+Optionally, one can overwrite the activation of a layer with their own function:
+```c++
+void set_activation(int index, std::function<float(float)> custom, std::function<float(float)> custom_deriv);
+```
+At this point could also specify a learning rate decay function as well by calling `init_decay`.
+```c++
+void init_decay(char* type, float a_0, float k);
+```
+Next, call `initialize()` to initialize the network's weights.
+Finally, train your network for one epoch with `train()`. Training one epoch at a time allows you control over the accuracy reporting (with functions like `get_cost()`, `get_accuracy()`, `get_val_cost()`, and `get_val_accuracy()`) and also allows effective use of services like W&B.
+
+### Examples
+In the `/scripts` directory there is a example of a neural network being used in conjuction with Weights & Biases, allowing for effective hyperparameter searches and accuracy reporting.
+
+## Compiling/Installing
### Precompiled Libraries
-Head to the 'Releases' section and pick up the right python library for you if you don't want to build from source.
+**Coming soon:**: Precompiled pyc files and binaries in 'Releases' section for you if you don't want to build from source.
+
+### Python Bindings
+
+1. Install both the C++ end of pybind11 and the python end.
+2. Build with your chosen configuration using `make`.
+3. Copy the `mrbpnn.cpython-37m-darwin.so` file into your personal project directory.
+4. Import `mrbpnn` from your Python code and use it.
+5. Look to the `example.py` file for simple usage of the library.
-### Build Configurations
+**Coming soon:** A guide on how to properly install Jacobian as a python library.
+**Coming soon:** Availability through package managers or a less convoluted install process.
-There are 5 build configurations, each one prioritizing program speed more than the last. *Warning!* Makefile assumes the presence of the Intel C++ Compiler (`icpc`) and the Intel Math Kernel Library. If you don't have this, replace all instances of `icpc` in the Makefile with `g++', remove all traces of the Math Kernel Library, and remove the flags `g++` doesn't understand.
+### Build Configurations (Building from Source)
+
+There are 5 build configurations, each one prioritizing program speed more than the last. *Warning!* The higher levels (>fast) are as of now unstable as changes to the code have caused it to not play nicely with MKL or icpc.
#### Level 1: `make`
Simply builds the project with no optimization at all. Use this if you don't want to wait long for the library to compile and don't care too much about speed in the moment.
@@ -60,15 +111,7 @@ This option is not implemented as of now.
Planned features include:
- Adding all the aforementioned compiler options as well as compiler options that are potentially unsafe.
- - Defining the `RECKLESS` macro which will skip anything that is not absolutely necessary in the code (with preprocessor statements like `#ifndef`).
-
-### Python Bindings
-
-1. Install both the C++ end of pybind11 and the python end.
-2. Build with your chosen configuration using `make`.
-3. Copy the `mrbpnn.cpython-37m-darwin.so` file into your personal project directory.
-4. Import `mrbpnn` from your Python code and use it.
-5. Look to the `example.py` file for simple usage of the library.
+ - Defining the `RECKLESS` macro which will skip anything that is not absolutely necessary in the code (with preprocessor statements like `#ifndef`). Will skip all sort of testing and checking before training.
## The Future
@@ -77,4 +120,4 @@ Jacobian is actively in development and the following are things that are planne
- Moving towards a more proper release by emphasizing usability.
- Further increasing speedups from a conceptual perspective with better algorithms, a implementation perspective with optimized code, and a low-level perspective with hardware optimizations + more compiler work.
- More advanced capabilities such as the inclusion of gradient descent optimizations.
-- More clear examples on how this is used as well as why one would use it.
+- Parallelization, data-oriented design, and more ways of increasing usability+speed.
