Issues BSD-3-Clause License CodeQL


Logo

Mojmelo

Report Bug · Request Feature

About The Project

The name Mojmelo is derived from the "Mojo Machine Learning" expression. It includes the implementation of Machine Learning algorithms from scratch in pure Mojo. Here is the list of the algorithms:

  • Linear Regression
  • Polynomial Regression
  • Logistic Regression
  • KNN
  • KMeans
  • HDBSCAN
  • DBSCAN
  • SVM
  • Naive Bayes
    1. GaussianNB
    2. MultinomialNB
  • Decision Tree (Regression/Classification)
  • Random Forest (Regression/Classification)
  • GBDT (Regression/Classification)
  • PCA

Preprocessing:

  • normalize
  • MinMaxScaler
  • StandardScaler
  • KFold
  • GridSearchCV
  • LabelEncoder

Documentation: https://yetalit.github.io/Mojmelo/docs/_index.html

Getting Started

If you are not familiar with Mojo projects, you can get started here: https://mojolang.org/docs/manual/get-started/

Prerequisites

  • mojo-compiler 1.0.0 or later

Optionally, bellow Python packages can be installed for a better usability and to run tests:

  1. Numpy
  2. Pandas
  3. Scikit-learn
  4. Matplotlib

Installation

There are three ways to install mojmelo: Using Pixi CLI, PyPI CLI and through the source code.

Additionally, completing the setup process (discussed later) is recommended.

Pixi CLI

Make sure you have the Modular community channel (https://repo.prefix.dev/modular-community) in your pixi.toml file in the channels section, then add mojmelo this way:

pixi add mojmelo

To start the setup process, run the following command from the main folder of your project:

bash ./.pixi/envs/default/etc/conda/test-files/mojmelo/0/tests/setup.sh

Note: If CPU cache details are available by the OS, benchmarking parts of the setup will be skipped. Otherwise, please try not to run other tasks on your pc during the process for better results.

PyPI CLI

Using the command below, the PyPI package containing the source code will be installed:

pip install mojmelo

Then start the setup process this way:

mojmelo-setup

Note: If CPU cache details are available by the OS, benchmarking parts of the setup will be skipped. Otherwise, please try not to run other tasks on your pc during the process for better results.

Source Code

Mojmelo can also be installed through the source code. This way, you will have the source code in your project.

First, Download mojmelo folder and setup.mojo file. To start the setup process, run these commands from where mojmelo folder and setup.mojo file are stored:

mojo build setup.mojo -o setup &&
./setup &&
./setup 1 &&
./setup 2 &&
./setup 3 &&
./setup 4 &&
./setup 5 &&
./setup 6 &&
./setup 7 &&
./setup 8 &&
./setup 9 &&
rm -f ./setup

Note: If CPU cache details are available by the OS, benchmarking parts of the setup will be skipped. Otherwise, please try not to run other tasks on your pc during the process for better results.

Usage

Importing models is straightforward:

from mojmelo.LinearRegression import LinearRegression

You may also want to use the utility codes written for this project:

from mojmelo.utils.Matrix import Matrix
from mojmelo.utils.utils import *

Here is an example code demonstrating a common training process:

from mojmelo.KNN import KNN
from mojmelo.utils.Matrix import Matrix
from mojmelo.preprocessing import train_test_split, GridSearchCV, LabelEncoder
from mojmelo.utils.utils import accuracy_score
from std.python import Python
import std.os as os

def main() raises:
    # Load the Iris dataset from scikit-learn using the Python interoperability API.
    var iris = Python.import_module("sklearn.datasets").load_iris()

    # Create a LabelEncoder instance.
    # This converts class labels into integer values that the model can work with.
    var le = LabelEncoder()

    # Convert the NumPy feature array into a native Matrix.
    var X = Matrix.from_numpy(iris.data)
    # Encode the target labels into integer values.
    var y = le.fit_transform(iris.target)

    # Define the hyperparameter values to test.
    # Here we evaluate KNN with k = 3, 5, and 7.
    var params = Dict[String, List[String]]()
    params["k"] = ["3", "5", "7"]
    # Find the best hyperparameters using grid search.
    # - accuracy_score is the evaluation metric.
    # - n_jobs=-1 uses all available CPU cores.
    #
    # GridSearchCV returns the best hyperparameters and their score. [0] contains the best parameters.
    var best_params = GridSearchCV[KNN](
        X,
        y,
        params,
        accuracy_score,
        cv=4,
        n_jobs=-1,
    )[0].copy()
    print("Tuned parameters:", best_params)

    # Split the dataset into training and testing sets.
    var X_train, X_test, y_train, y_test = train_test_split(
        X,
        y,
        test_size=0.2,
        random_state=1234,
    )

    # Create a KNN model using the best hyperparameters found above.
    var knn = KNN(best_params)
    # Train the model using the training data.
    knn.fit(X_train, y_train)
    # Save the trained model to disk.
    knn.save("knn")

    # Load the saved model back from disk.
    knn = KNN.load("knn")
    # Predict the labels for the test samples.
    var y_pred = knn.predict(X_test)
    # Compare the predictions with the expected labels.
    print("KNN classification accuracy:", accuracy_score(y_test, y_pred))

    # Remove the saved model file created by this example.
    os.remove("knn.mjml")

More examples are available in tests folder.

Benchmarks (AMD Zen 4)

KMeans

Model Fit Time (s) ARI vs sklearn ARI vs truth
sklearn KMeans 0.2764 ± 0.0010 - 0.9390
mojmelo KMeans 0.1713 ± 0.0029 0.8822 0.9390

HDBSCAN (algorithm='boruvka_kdtree')

Model Fit Time (s) ARI vs sklearn ARI vs fast_hdbscan ARI vs truth
skl-contrib HDBS 1.1945 ± 0.0067 - - 0.9988
fast hdbscan 0.2412 ± 0.0014 - - 0.9989
mojmelo HDBS 0.1765 ± 0.0032 0.9923 0.9989 0.9933

DBSCAN (algorithm='kd_tree')

Model Fit Time (s) ARI vs sklearn ARI vs truth
sklearn DBS 1.0625 ± 0.0020 - 0.8605
mojmelo DBS 0.4817 ± 0.0035 1.0000 0.8605

KNN (algorithm='kd_tree')

Model Fit Time (s) Predict Time (s) Accuracy
sklearn KNN 0.0353 ± 0.0005 1.7600 ± 0.0063 0.8543
mojmelo KNN 0.0149 ± 0.0006 0.2126 ± 0.0040 0.8347

SVM

Model Fit Time (s) Predict Time (s) Accuracy
sklearn SVM 1.2857 ± 0.0020 0.3720 ± 0.0008 0.9750
mojmelo SVM 0.8618 ± 0.0091 0.0600 ± 0.0002 0.9750

DecisionTreeClassifier

Model Fit Time (s) Predict Time (s) Accuracy
sklearn DTC 0.8498 ± 0.0033 0.0004 ± 0.0000 0.9667
mojmelo DTC 0.0810 ± 0.0057 0.0001 ± 0.0000 0.9668

DecisionTreeRegressor

Model Fit Time (s) Predict Time (s) MSE
sklearn DTR 0.6466 ± 0.0006 0.0005 ± 0.0000 8247.9358
mojmelo DTR 0.0795 ± 0.0049 0.0003 ± 0.0000 8192.1982

RandomForestClassifier

Model Fit Time (s) Predict Time (s) Accuracy
sklearn RFC 0.4401 ± 0.0078 0.0139 ± 0.0002 0.9163
mojmelo RFC 0.4064 ± 0.0036 0.0044 ± 0.0001 0.9144

RandomForestRegressor

Model Fit Time (s) Predict Time (s) MSE
sklearn RFR 2.0257 ± 0.0050 0.0134 ± 0.0004 8454.5517
mojmelo RFR 1.2247 ± 0.0094 0.0067 ± 0.0002 9155.6895

PCA (svd_solver='full')

Model Fit Time (s) Transform Time (s) Explained Var
sklearn PCA 0.2359 ± 0.0081 0.0088 ± 0.0019 0.5375
mojmelo PCA 0.0515 ± 0.0027 0.0103 ± 0.0001 0.5375

Contributing

Contributions are what make the open source community such an amazing place to learn, inspire, and create. Any contributions you make are greatly appreciated.

Contributions can be done to the project in these 3 ways:

  1. Applying improvements to the code and opening a Pull Request
  2. Reporting a bug
  3. Suggesting new features

Acknowledgments

  • Mojo usage and distribution are licensed under the Modular Community License.

  • Libsvm, A Library for Support Vector Machines by Chih-Chung Chang and Chih-Jen Lin licensed under the BSD-3-Clause license.

  • HDBSCAN implementation is partially based on hdbscan by Leland McInnes, John Healy and Steve Astels licensed under the BSD-3-Clause license and Fast Multicore HDBSCAN by Tutte Institute for Mathematics and Computing licensed under the BSD-2-Clause license.

  • matmul implementation is based on matmul.mojo by Ethan Wu (YichengDWu) licensed under the Apache-2.0 license.

  • argmin, argmax and argsort implementations and utils.algorithm submodule are based on codes from Modular licensed under the Apache License v2.0 with LLVM Exceptions.

  • KDTREE2, a kd-tree implementation in Fortran 95 and C++ by Matthew B. Kennel.

  • Initially drew inspiration from Patrick Loeber's MLfromscratch.

Edit this page