Additional evaluation metrics


#1

Hi there! I m using DLS to design and train mostly classification DL models and I have to say that the process is as intuitive as it gets. The layer options cover pretty much everything!

I was wondering if there’s somehow an option to extract additional metrics for the model (apart from train/test accuracy and loss). It would be really helpful as a feature, since there are cases where accuracy doesn’t provide sufficient info w.r.t FP/FN.
Thank you for this awesome work!


#2

Hey
Right now we don’t have any option for calculating matrix but we will surely consider this feature.
What you can do now is that you can run the inference on the dataset and download the CSV file of the results from which you can calculate FP/FN easily.

Regards
Rajat


#3

Thank you. I was considering the option you mentioned, however I shouldn’t run inference on the whole dataset, only the test partition, otherwise I wont be able to distinguish between train-test FP/FN.
Since I don’t think I can split the dataset using the same random seed that DLS used, I could download it whole, split it to train-test percentage (using any random seed) and evaluate only the test one. It’s not right, but it’s still better than nothing.


#4

You can do one thing is that split your data between train and test.
Shuffle the training data before uploading it on DLS then uncheck the shuffle data option.
Train your model and check inference on your validation data and testing data by uploading the test dataset on inference tab.
Test data zip file should be similar to your training data i.e. same data name in the CSV file.
You just need to make 1 change that is name your csv file as “test.csv” instead of “train.csv”

Regards
Rajat


#5

Excellent trick! Thanks :slight_smile: