I have a web application that needs to use scraping, and I need to decide between 3 different architectures on how the scraping will be divided between the backend and frontend.
My web application uses a NodeJS backend (with Express) and a React frontend. My goal is to analyze/scrape some HTML from an external website (we'll call it fakewebsite.com), probably with a package like Cheerio.
Here is the general flow that my application needs:
User specifies a url from fakewebsite.com (ex. fakewebsite.com/post/12345) on my front end website
That page is analyzed and specific fields are scraped from its HTML, eventually being given to backend to perform further backend logic
I can think of 3 possible configurations to achieve this goal. I need to decide which one is the best to move forward with.
Method A: All Front-end
This seems nice, but would expose some business logic (not critical logic, but also not ideal) to front end users.
Method B: All back-end
I feel like this option would not scale because fakewebsite.com would block my backend from performing requests after a while
Method C: Mix of front end and backend
I like this option the best. But, I am hesitant about security flaws with this approach, since in theory a malicious user could use the post endpoint with some HTML designed to inject malicious code into my Node server.
Are my concerns about security flaws in Method C (uploading HTML to Express) justified? Do you have any recommendations on the best way to proceed with architecting this scraping problem?
Just make sure not to post any potentially malicious code to your backend.
You can use a sanitization library like 'sanitize-html' (https://npmjs.com/package/sanitize-html) or 'dompurify' (https://www.npmjs.com/package/dompurify), and sanitize your string before you POST it. Then you'll be fine.