Skip to content

Parse the data from JSON file without loading the whole JSON file #97

@machineCYC

Description

@machineCYC

Since I have a big JSON file(20GB), I want to parse the data from that JSON file, but I do not want to load the whole JSON file

So, I only have the JSON file path, and I want to get the specific data in JSON file

such as below is a JSON file, I want to parse the "ColC" data

data = {
    "ColA":"XXX",
    "ColA":"XXX",
    "ColB":"XXX",
    "ColC":[
        {"ColC1":"ZZZ1", "time":"TTT1"},
        {"ColC2":"ZZZ2", "time":"TTT2"},
        {"ColC3":"ZZZ3", "time":"TTT3"},
        {"ColC4":"ZZZ4", "time":"TTT4"},
        ...
        {"ColC100000":"ZZZ100000", "time":"TTT100000"},
    ],
    "ColD":[
        {"ColD1":"ZZZ1", "time":"TTT1"},
        {"ColD2":"ZZZ2", "time":"TTT2"},
        {"ColD3":"ZZZ3", "time":"TTT3"},
        {"ColD4":"ZZZ4", "time":"TTT4"},
        ...
        {"ColD100000":"ZZZ100000", "time":"TTT100000"},
    ],
    "ColE":"XXX",
    "ColF":"XXX",
    ....
    "ColZ":"XXX",
}

I am not sure python-jsonpath-rw can do this or not,

have any examples can share with me?

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions