Parquet

Published by onesixx on 23-06-0923-06-09

https://parquet.apache.org/ : Apache Parquet

https://arrow.apache.org/docs/python/parquet.html#

표준화된 오픈소스 컬럼형식의 저장소 포멧

padas.dataFrame <–> parquet

import pandas as pd

# conda install -c conda-forge fastparquet

# pip install fastparquet
# pip install pyarrow
import pyarrow as pa
import pyarrow.parquet as pq


df = pd.read_csv('dataset/st.csv')
#isinstance(trn_df, pd.DataFrame)  # True

### pd.DataFrame to parquet
df.to_parquet('dataset/st.parquet')
# pq.write_table(pa.Table.from_pandas(trn_df), 'dataset/st1.parquet')

train_df = pd.read_parquet('dataset/st.parquet')
# df = pq.read_table('./dataset/st1.parquet').to_pandas()

Parquet

onesixx

Uncategorized

무환수 어항

Uncategorized

copilot

Uncategorized

inference demo

Parquet

onesixx

Related Posts

Uncategorized

무환수 어항

Uncategorized

copilot

Uncategorized

inference demo