For reproduction of the dataset, run dataset_creator.py with previously installed dependencies.

For adding your own data, either directly use the API provided by semanticscholar to retrieve dataset entries or create them from files preprocessed using GROBID with dataset_creator.py and uncomment the preparation steps.

dataset_creator.py also requires the following commands to be run after installation of requirements:

python -m spacy download en_core_web_sm
python -c "import nltk; nltk.download('stopwords')"


A fully documented creation pipeline including processing using GROBID as well as the semanticscholar API will be made publicly available once this work is published.
