Business
Jobs
  • About Us
  • Solutions
    • Job Postings
      Post your job and receive qualified candidates in 48h.
    • Candidate Assessments
      500+ technical and psychological tests, plus anti-fraud.
    • Headhunting
      Tailor-made executive search from start to finish.
    • Payroll + EOR
      Payroll dispersal and EOR across 15+ LATAM countries.
  • Pricing
  • Jobs

0

247
Views
AWS EMR: do i need core nodes if I am planning to use EMRFS

I am going through the online documentation and I found following diference between core not and task node.

  1. Core node has hdfs while task node does not have HDFS.

due to above, AWS suggest it's not a good idea to scale core nodes based on load as hdfs re-balancing could take time and should re-balance task nodes only.

However, if I am planning to use EMRFS, do i need core nodes? what is the user of HDFS in this case if I am planning to access data from s3.

over 4 years ago · Santiago Trujillo
1 answers
Answer question

0

You need at least 1 Core Node.

If you want to use s3distcp after finish a job writing to local HDFS, then you need more such Core Nodes.

over 4 years ago · Santiago Trujillo Report
Answer question
Find remote jobs

Discover the new way to find a job!

Top jobs
Top job categories
Business
Post vacancy Pricing Sales
Legal
Terms and conditions Privacy policy
© 2026 PeakU Inc. All Rights Reserved.
Andres GPT
Show me some job opportunities
There's an error!