Khurrum Nasim
2016-06-01 15:14:27 UTC
Hello All,
Seeking some advice regarding the following:
I have a JSON ETL task. You know we all done some ETL in our lives before - extract data, apply some transformation to it, and load it back.
I have a fairly huge amount of JSON that I need to iterate over and check for the existence of a particular <key, value> combination lets call it for argument sake the “target”.
If the “target" is found and the value is not null, I have to create a duplicate of it but with a distinct name for the key. If the “target” is found but has null value/no value, then simply create a new key leaving the value null.
Because JSON can have a deep nested structure a recursive routine wouldn’t be ideal here to tackle the problem given that I don’t know how large the data is.
As I said earlier I have no idea how large the data is ? It could be a large set possibly several gigs. Let’s assume 50-100 gig.
Thanks,
Khurrum
Seeking some advice regarding the following:
I have a JSON ETL task. You know we all done some ETL in our lives before - extract data, apply some transformation to it, and load it back.
I have a fairly huge amount of JSON that I need to iterate over and check for the existence of a particular <key, value> combination lets call it for argument sake the “target”.
If the “target" is found and the value is not null, I have to create a duplicate of it but with a distinct name for the key. If the “target” is found but has null value/no value, then simply create a new key leaving the value null.
Because JSON can have a deep nested structure a recursive routine wouldn’t be ideal here to tackle the problem given that I don’t know how large the data is.
As I said earlier I have no idea how large the data is ? It could be a large set possibly several gigs. Let’s assume 50-100 gig.
Thanks,
Khurrum