Classification Models Fine-Tuning

In this example, we again use our sample PostgreSQL database.

First, we create and train the model using a subset of the customer_churn data, considering only female customers.

CREATE MODEL mindsdb.adjust_customer_churn_model
FROM example_db
    (SELECT *
    FROM demo_data.customer_churn
    WHERE gender = 'Female')
PREDICT churn;

On execution, we get:

Query successfully completed

We can check its status using this command:

SELECT *
FROM mindsdb.models
WHERE name = 'adjust_customer_churn_model';

Once the status is complete, we can query for predictions.

SELECT churn, churn_explain
FROM mindsdb.adjust_customer_churn_model
WHERE seniorcitizen = 0
AND partner = 'Yes'
AND dependents = 'No'
AND tenure = 1
AND phoneservice = 'No'
AND multiplelines = 'No phone service'
AND internetservice = 'DSL';

On execution, we get:

+--------+------------------------------------------------------------------------------------------------------------------------------------------------------------+
| churn  | churn_explain                                                                                                                                              |
+--------+------------------------------------------------------------------------------------------------------------------------------------------------------------+
| No     | {"predicted_value": "No", "confidence": 0.9887640449438202, "anomaly": null, "truth": null, "probability_class_No": 0.934, "probability_class_Yes": 0.066} |
+--------+------------------------------------------------------------------------------------------------------------------------------------------------------------+

Let’s adjust this model with more training data. Now we also consider male customers.

FINETUNE mindsdb.adjust_customer_churn_model
FROM example_db
    (SELECT *
    FROM demo_data.customer_churn
    WHERE gender = 'Male');

While the model is being generated and trained, it is not active. The model becomes active only after it completes generating and training.

To check the status and versions of the model, run this command:

SELECT name, engine, project, active, version, status
FROM mindsdb.models_versions
WHERE name = 'adjust_customer_churn_model';

On execution, we get:

+-----------------------------+-----------+---------+--------+---------+----------+
| name                        | engine    | project | active | version | status   |
+-----------------------------+-----------+---------+--------+---------+----------+
| adjust_customer_churn_model | lightwood | mindsdb | false  | 1       | complete |
| adjust_customer_churn_model | lightwood | mindsdb | true   | 2       | complete |
+-----------------------------+-----------+---------+--------+---------+----------+

Let’s query for a prediction again.

SELECT churn, churn_explain
FROM mindsdb.adjust_customer_churn_model
WHERE seniorcitizen = 0
AND partner = 'Yes'
AND dependents = 'No'
AND tenure = 1
AND phoneservice = 'No'
AND multiplelines = 'No phone service'
AND internetservice = 'DSL';

On execution, we get:

+--------+--------------------------------------------------------------------------------------------------------------------------------------------------------------+
| churn  | churn_explain                                                                                                                                                |
+--------+--------------------------------------------------------------------------------------------------------------------------------------------------------------+
| No     | {"predicted_value": "No", "confidence": 0.9887640449438202, "anomaly": null, "truth": null, "probability_class_No": 0.9294, "probability_class_Yes": 0.0706} |
+--------+--------------------------------------------------------------------------------------------------------------------------------------------------------------+

Here after adjusting the model, there are no significant changes to the predictions. However, the probability class for Yes and No values has been updated. The probability of a Yes value has increased slightly, while the probability of a No value has decreased.