Quantcast
Channel: Active questions tagged python - Stack Overflow
Viewing all articles
Browse latest Browse all 23131

Converting Specific Data Segments from a .dat File to CSV Format with Timestamps

$
0
0

I have a .dat file containing data segments structured in a format similar to the one below:

$GPS LAT GPS LONG   ROLL  PITCH AZIMUT   T IN  PmBARS  T OUT    RH%8.9636   77.7201   -0.5    1.0    0.0   43.3   998.7   25.1   86.0                       BL#  MONTH   DAY  YEAR  HOUR   MIN  VAL1  VAL2  VAL3  VAL4 19753    12     7  2023     0     0    73    82   121     0  SPU1  SPU2  SPU3  SPU4 NOIS1 NOIS2 NOIS3 NOIS4 FEMAX SOFTW     1     0     0     0  4203  4102  4700    18   503  9065  FE11  FE12  FE21  FE22  SNR1  SNR2  SNR3  SNR4 CHECK   JAM     8     7     8     8   139   139   139     0    40 11100   ALT    CT SPEED   DIR     W    SW    SU    SV  ETAM   200    46   299    46     0     1    19    13     0   190    52   288    46     0     1    18    15     0   180    58 -9999    47     0     1    18    17     0   ...   ...   ...   ...   ...   ...   ...   ...   ...   30    975    29    73    -8     1    14     2     0 $  GPS LAT  GPS LONG   ROLL  PITCH AZIMUT   T IN  PmBARS  T OUT    RH%    8.9636   77.7201   -1.0    0.9    0.0   48.5   997.7   28.2   73.2                       BL# MONTH   DAY  YEAR  HOUR   MIN  VAL1  VAL2  VAL3  VAL4 19944    12     7  2023    0     10    21    34    51     0  SPU1  SPU2  SPU3  SPU4 NOIS1 NOIS2 NOIS3 NOIS4 FEMAX SOFTW     1     0     0     0  5502  5403  5701     8   490  9065  FE11  FE12  FE21  FE22  SNR1  SNR2  SNR3  SNR4 CHECK   JAM     8     7     9     7   135   136   139     0    56 11100   ALT    CT SPEED   DIR     W    SW    SU    SV  ETAM   200    66   598    39    -9    16    22    36   169   190    72   599    39    -9    16    22    36   189   180    77   600    38    -9    17    22    36   194    ..    ..   ...    ..     .    ..    ..    ..   ...    20    80   600    37    -9    15    22    35   154    30    83   591    36    -9    14    22    35   115$Again the same above format only the data will change. This dataset contains multiple segments similar to the provided snippet, each representing a 10-minute interval of records between each $.$

I need assistance in converting this data into a CSV file named 'output.csv' with a specific format:

  1. The first row contains headers specified as below.
  2. Subsequent rows should contain:
  • Timestamp in YYYY-MM-DD HH:MM:SS format extracted from the XML.
  • Wind speeds at different altitudes (200m to 30m).
  • Wind directions at different altitudes (200m to 30m).

For example, for a given input, the desired output CSV would resemble:

Timestamp, ALT200_Speed, ALT190_Speed, ..., ALT30_Speed, ALT200_Dir, ALT190_Dir, ..., ALT30_Dir2023-12-07 00:00:00, 299, 288, ..., 29, 46, 46, ..., 732023-12-07 00:00:10, 598, 599, ..., 591, 39, 39, ..., 36

I seek guidance or a code solution in Python or any other suitable language to efficiently process this large dataset and generate the desired output CSV containing the specified information.

Read .dat file and extract relevant data segments

Loop through the file, identify and extract segments containing wind data between $ symbol

Process each segment to extract timestamps, wind speeds, and directions

Generate 'output.csv' with appropriate headers and write data

Write each timestamp, wind speeds, and directions to the CSV file in the specified format


Viewing all articles
Browse latest Browse all 23131

Trending Articles



<script src="https://jsc.adskeeper.com/r/s/rssing.com.1596347.js" async> </script>