ValueError when loading YAML

Hi, I’m trying to make predictions locally on PyCharm. I’ve already downloaded the trained model but the IDE throws me the following ValueError while loading the YAML file:

(I’ve also tried defining the model manually and then loading the .h5 file; it works but the predictions are made randomly)

ValueError: Improper config format: {‘data’: {‘dataset’: {‘name’: ‘hangouts-augmented3’, ‘samples’: 4990, ‘type’: ‘private’}, ‘datasetLoadOption’: ‘batch’, ‘kfold’: 1, ‘mapping’: {‘categoria_conducta’: {‘options’: {‘Normalization’: False, ‘Scaling’: 1}, ‘port’: ‘Ignore’, ‘shape’: ‘’, ‘type’: ‘Numeric’}, ‘categoria_reaccion’: {‘options’: {}, ‘port’: ‘OutputPort0’, ‘shape’: ‘’, ‘type’: ‘Categorical’}, ‘mensaje’: {‘options’: {‘Normalization’: False, ‘Scaling’: 1}, ‘port’: ‘InputPort0’, ‘shape’: ‘’, ‘type’: ‘Array’}}, ‘numPorts’: 1, ‘samples’: {‘split’: 4, ‘test’: 499, ‘training’: 3992, ‘validation’: 499}, ‘shuffle’: True}, ‘model’: {‘connections’: [{‘source’: ‘Activation_7’, ‘target’: ‘MaxPooling1D_3’}, {‘source’: ‘Convolution1D_7’, ‘target’: ‘BatchNormalization_14’}, {‘source’: ‘Reshape_1’, ‘target’: ‘Convolution1D_1’}, {‘source’: ‘Dropout_105’, ‘target’: ‘Dense_17’}, {‘source’: ‘Dense_17’, ‘target’: ‘Dropout_13’}, {‘source’: ‘Dense_16’, ‘target’: ‘Dropout_12’}, {‘source’: ‘BatchNormalization_46’, ‘target’: ‘Activation_1’}, {‘source’: ‘MaxPooling1D_3’, ‘target’: ‘Flatten_3’}, {‘source’: ‘Activation_1’, ‘target’: ‘Convolution1D_7’}, {‘source’: ‘Dense_18’, ‘target’: ‘Output_3’}, {‘source’: ‘Convolution1D_1’, ‘target’: ‘BatchNormalization_46’}, {‘source’: ‘Dense_150’, ‘target’: ‘Dropout_105’}, {‘source’: ‘Dropout_13’, ‘target’: ‘Dense_18’}, {‘source’: ‘Dropout_12’, ‘target’: ‘Dense_150’}, {‘source’: ‘Flatten_3’, ‘target’: ‘Dense_16’}, {‘source’: ‘BatchNormalization_14’, ‘target’: ‘Activation_7’}, {‘source’: ‘Input_1’, ‘target’: ‘Reshape_1’}], ‘layers’: [{‘args’: {}, ‘class’: ‘Input’, ‘name’: ‘Input_1’, ‘x’: 35.360000000000014, ‘y’: 47.94}, {‘args’: {‘filter_length’: ‘2’, ‘nb_filter’: ‘16’}, ‘class’: ‘Convolution1D’, ‘name’: ‘Convolution1D_1’, ‘x’: 31.990000000000002, ‘y’: 293.21999999999997}, {‘args’: {‘target_shape’: ‘(300, 1)’}, ‘class’: ‘Reshape’, ‘name’: ‘Reshape_1’, ‘x’: 34.60000000000001, ‘y’: 165.51}, {‘args’: {‘activation’: ‘relu’}, ‘class’: ‘Activation’, ‘name’: ‘Activation_1’, ‘x’: 29.61, ‘y’: 486.65999999999997}, {‘args’: {‘filter_length’: ‘2’, ‘nb_filter’: ‘8’}, ‘class’: ‘Convolution1D’, ‘name’: ‘Convolution1D_7’, ‘x’: 29.94999999999999, ‘y’: 603.35}, {‘args’: {}, ‘class’: ‘BatchNormalization’, ‘name’: ‘BatchNormalization_14’, ‘x’: 33.94999999999999, ‘y’: 690.35}, {‘args’: {‘activation’: ‘relu’}, ‘class’: ‘Activation’, ‘name’: ‘Activation_7’, ‘x’: 39.56, ‘y’: 785.01}, {‘args’: {}, ‘class’: ‘MaxPooling1D’, ‘name’: ‘MaxPooling1D_3’, ‘x’: 346.24, ‘y’: 53.25999999999999}, {‘args’: {}, ‘class’: ‘Flatten’, ‘name’: ‘Flatten_3’, ‘x’: 342, ‘y’: 160}, {‘args’: {‘activation’: ‘relu’, ‘output_dim’: ‘1000’}, ‘class’: ‘Dense’, ‘name’: ‘Dense_16’, ‘x’: 347.35, ‘y’: 316.72}, {‘args’: {‘p’: ‘0.5’}, ‘class’: ‘Dropout’, ‘name’: ‘Dropout_12’, ‘x’: 350.35, ‘y’: 408.72}, {‘args’: {‘activation’: ‘relu’, ‘output_dim’: ‘100’}, ‘class’: ‘Dense’, ‘name’: ‘Dense_17’, ‘x’: 675.33, ‘y’: 512.28}, {‘args’: {‘p’: ‘0.5’}, ‘class’: ‘Dropout’, ‘name’: ‘Dropout_13’, ‘x’: 674.33, ‘y’: 600.28}, {‘args’: {‘activation’: ‘softmax’, ‘output_dim’: ‘4’}, ‘class’: ‘Dense’, ‘name’: ‘Dense_18’, ‘x’: 665.87, ‘y’: 695.91}, {‘args’: {}, ‘class’: ‘Output’, ‘name’: ‘Output_3’, ‘x’: 668.55, ‘y’: 793.1899999999999}, {‘args’: {}, ‘class’: ‘BatchNormalization’, ‘name’: ‘BatchNormalization_46’, ‘x’: 27, ‘y’: 394}, {‘args’: {‘activation’: ‘relu’, ‘output_dim’: ‘500’}, ‘class’: ‘Dense’, ‘name’: ‘Dense_150’, ‘x’: 352.35, ‘y’: 525.72}, {‘args’: {‘p’: ‘0.5’}, ‘class’: ‘Dropout’, ‘name’: ‘Dropout_105’, ‘x’: 357.35, ‘y’: 620.72}]}, ‘params’: {‘advance_params’: True, ‘batch_size’: 32, ‘is_custom_loss’: False, ‘loss_func’: ‘categorical_crossentropy’, ‘num_epoch’: 100, ‘optimizer’: {‘name’: ‘Adadelta’}}, ‘project’: ‘prueba-cnn’}

Hi @franciscoserrano

I think this YAML is designed especially for DLS, not for PyCharm. YAML is the general idea how to store models, but not unified standard across all IDEs and SDKs…

Hi, thanks for the quick answer.

I’ve compared the DLS YAML against a Keras YAML and indeed, you were right. They are totally different.

Is there any workaround for running the trained NN locally in PyCharm? I find this tool extremely useful as I’m starting to getting involved into Deep Learning. Perhaps it would be a nice feature to include in future releases

For my best knowledge for now there is no such workaround, maybe DLS Team will add such feature in future releases. Basically such feature can be done with python scripting, translating between formats, all you need - description of desired formats, format reader and writer.

1 Like

@franciscoserrano, the config.yaml file in the downloaded trained model is DLS project configuration. That is used to import the project back into DLS.

If your goal is to run the inference locally in PyCharm, then you should be able to load model.h5 using Keras’s load_model api. model.h5 contains weights as well as the Keras model. So there is no need for you to manually create Keras’s model.

Downloaded trained model contains a reference implementation in test.py which shows how to use the the trained model.

Hi @rajendra

Thanks for your reply. I tried proceeding as you said, manually coding the model and then loading the .h5 file. It ran perfectly, but the accuracy was not as expected. While in DLS I got 90/91 percent of validation accuracy (data shuffled), when testing the model with a Scikit Learn test split, the accuracy was below 10 percent. It was random guess.

I spent the whole day trying to figure out what was the mistake, but without success. Perhaps the weights didn’t load correctly, but in that case Python would have thrown some error I think.

Hi @franciscoserrano

You dont need to create the model’s manually. Following should be sufficient to load the trained model

from keras.models import load_model

model = load_model ("model.h5") 

Lets first try to run reference implementation (test.py in downloaded trained mode)l with you test dataset. You can run test.py as follows from terminal:

python test.py <full path the test.csv>

This will generate test_result.csv with the prediction.

What accuracy do you see in this test_result.csv ?

Are you using pre-trained model ? There is a known issue with pre-trained model with imagenet weights where saved weights are not correct.

Hi @rajendra

I didn’t try the test.py before because I thought that it worked only on Linux, but I tried on Windows and worked perfectly. The accuracy in the .csv was the same as shown in DLS. No surprises.

I have to recognize that the problem had nothing to do with DLS. The problem was my inexperience with Keras. I used model.evaluate(…) to obtain the test accuracy, but I didn’t take into account that the last layer of my model was a Softmax layer. It outputs a vector of probabilities, which almost never matched with the labels, because their format wasn’t a vector of probabilities but a one-hot vector.

Then I tried with model.predict(…) and indeed the predictions were the same as in test_result.csv, so fortunately the problem was solved.

Regards to using in PyCharm the model trained on DLS, by loading the .h5 is enough as you said.

Thanks a lot for your help!!

Saludos