Hello, I ran in to C error and not really sure how to remedy this.
I started initial testing of my project with csv file containing 2 columns with array in each one.
I validated that dataset csv file is clean and contains no empty or null values.
Things work fine with 7million lines in the training set(500mb) but as I go over that training trows out of memory error.
Test server is AMD epyc with 88gb of ram, used in CPU mode.
I tried loading data via file manager, nothing makes any difference. This appears to be an issue with batch size.
I want to train on datasets that will be 80-100GB so hitting issues at 1GB is problematic.