Quantcast
Channel: Active questions tagged python - Stack Overflow
Viewing all articles
Browse latest Browse all 23131

Apache Beam - AttributeError: Error trying to access nonexistent attribute `0` in write result

$
0
0

**I want to ingest data into bigquery using following code and it is giving me following errors**I am very new to apache beam so help me to get better understanding.

File "C:\Users\Stranger\OneDrive - Stranger\Desktop\Coding\POC2\beam_bq.py", line 106, in poc1()File "C:\Users\Stranger\OneDrive - Stranger\Desktop\Coding\POC2\beam_bq.py", line 99, in poc1result = (errors | 'PrintErrors'>> beam.Map(print))~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~File "C:\Users\Stranger\OneDrive - Stranger\Desktop\Coding\POC2\Lib\site-packages\apache_beam\transforms\ptransform.py", line 1110, in rorreturn self.transform.ror(pvalueish, self.label)^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^File "C:\Users\Stranger\OneDrive - Stranger\Desktop\Coding\POC2\Lib\site-packages\apache_beam\transforms\ptransform.py", line 616, in rorreplacements = {^File "C:\Users\Stranger\OneDrive - Stranger\Desktop\Coding\POC2\Lib\site-packages\apache_beam\transforms\ptransform.py", line 617, in id(v): p | 'CreatePInput%s' % ix >> Create(v, reshuffle=False)^^^^^^^^^^^^^^^^^^^^^^^^^^File "C:\Users\Stranger\OneDrive - Stranger\Desktop\Coding\POC2\Lib\site-packages\apache_beam\transforms\core.py", line 3723, in init
self.values = tuple(values)^^^^^^^^^^^^^File "C:\Users\Stranger\OneDrive - Stranger\Desktop\Coding\POC2\Lib\site-packages\apache_beam\io\gcp\bigquery.py", line 2501, in getitemraise AttributeError(AttributeError: Error trying to access nonexistent attribute 0 in write result. Please see documentation for available attributes.

def poc1():    import apache_beam as beam    # Create pipeline.    schema = ({'fields': [{'name': 'a', 'type': 'STRING', 'mode': 'REQUIRED'}]})    pipeline = beam.Pipeline()    errors = (pipeline                | 'Data'>> beam.Create([2, 2])                | 'CreateBrokenData'>> beam.Map(lambda src: {'a': src} if src == 2 else {'a': '2'})                | 'WriteToBigQuery'>> beam.io.WriteToBigQuery(                    table='dummy_a_table',                    dataset='PM_INGEST_TEMP',                    project="bmas-eu-digi-pipe-dev",                    schema=schema,                    create_disposition=beam.io.BigQueryDisposition.CREATE_IF_NEEDED,                    write_disposition=beam.io.BigQueryDisposition.WRITE_TRUNCATE,                    custom_gcs_temp_location='gs://destination_bucket_final'                    # insert_retry_strategy='RETRY_ON_TRANSIENT_ERROR',                    # create_disposition='CREATE_IF_NEEDED',                    # write_disposition='WRITE_APPEND')                    )                )    result = (errors | 'PrintErrors'>> beam.Map(print))if __name__ == '__main__':    # run()    poc1()

I want to ingest data into bigquery using code and it is giving me errors. I am using windows 10, python 3.10 and want to run locally not on google cloud dataflow.


Viewing all articles
Browse latest Browse all 23131

Trending Articles