I'm trying to fetch data from multiple raw .md files from Guthub repo. Currently I'm able to fetch only one, yet I need to get to all of them.
I have a github repo and Im looking to fetch data from raw .md file, which is not a problem. The problem is that the repo has bunch of folders and each folder has its own .md file. I need to make some sort of map through all folders and fetch all of the .md files.
Lets say I have a github repo with following folders:
folder1 -> text1.md
folder2 -> text2.md
folder3 -> text3.md
I'm currently being able to fetch only one raw md usinng the following method
let fetchData = () => {
axios.get("https://raw.githubusercontent.com/user-name/repo-name/master/folder1/text1.md").then(response => {
console.log(response)
}).catch(error => {
console.log(error)
})
}
My goal is to fetch all text1, text2, text3.md so I can map through them and display in the table
Based on your comments, I would say that your best bet is to make a node worker that you can run weekly (o during deployments) that would crawl information form folders (filename and content) you tell pass it to, and then saved that information in some way you can later consume from gatsby (I guess the ideal way would be to put it on gatsby GraphQL).
This is a vague idea on how that worker could be, with the too limited information I have:
let repoBaseUrl = 'https://raw.githubusercontent.com/user-name/repo-name/master/';
let folders = [
'folder1',
'folder2',
'folder3'
];
let fetchFileName = async (folder) => {
// Your function to get the filename
return filename;
}
let fetchFileContent = async (folder, filename) => {
try {
const response = await axios.get(`${repoBaseUrl}${folder}${filename}`);
return response.data;
}
catch(error) {
// do something with the error
}
}
let fetchFolderContent = async () => {
const data = {};
folders.forEach(async (folder) => {
const filename = await fetchFileName(folder);
const content = await fetchFileContent(folder, filename);
data[folder] = {
filename,
content,
}
});
return data;
}
let main = async () => {
const data = await fetchFolderContent();
// Process your data
// IE: save it GraphQL so you can consume it from Gatsbt
}
main();