How can I prepare my data?


#1

So I have a large ~1 GB CSV file with forex tick price data for a whole year, formatted like below:

My question is, how can I wrangle with this data in the following ways:

  • Cut out 10000 randomized 30 minutes sections of the data.
  • For every 30 minutes section, make simulated Long AND Short trades at the 6 minute mark, with a certain predetermined stop loss and take profit. Then label each piece of 30 minute section into one of 4 categories: Should go long, Should go short, Both are profitable (unlikely, but let’s list it out anyway), and Either is okay.
  • Generate timeseries data of the first 5 minute of every 30 minute section in a format that an LSTM neural network in DLS can process, with correct labeling and everything.

#2

Hello you can’t do this directly with DLS for this you have to write a python script if you want I can help you with that.
Meanwhile you can refer to this article for creating Time series data for LSTM

Regards
Rajat


#3

Thanks for the reply. And yes I’ll greatly appreciate your help!
So the Excel file in the article is how my data should look? I thought my data should have looked like the arrays in the IMDB data set.
EDIT: Also does the prepared data needs to have a uniform data interval? For example, do we need to explicitly list out everything that happens during a millisecond even if nothing actually happens?