WEB开发网
开发学院软件开发Java hadoop mapred-queue-acls 配置 阅读

hadoop mapred-queue-acls 配置

 2012-05-22 15:50:50 来源:WEB开发网   
核心提示: hadoop作业提交时可以指定相应的队列,例如:-Dmapred.job.queue.name=queue2通过对mapred-queue-acls.xml和mapred-site.xml配置可以对不同的队列实现不同用户的提交权限.先编辑mapred-site.xml,hadoop mapred-queue-a

  hadoop作业提交时可以指定相应的队列,例如:-Dmapred.job.queue.name=queue2
通过对mapred-queue-acls.xml和mapred-site.xml配置可以对不同的队列实现不同用户的提交权限.
先编辑mapred-site.xml,修改配置如下(增加四个队列):
<property>
<name>mapred.queue.names</name>
<value>default,queue1,queue2,queue3,queue4</value>
<description> Comma separated list of queues configured for this jobtracker.
Jobs are added to queues and schedulers can configure different
scheduling properties for the various queues. To configure a property
for a queue, the name of the queue must match the name specified in this
value. Queue properties that are common to all schedulers are configured
here with the naming convention, mapred.queue.$QUEUE-NAME.$PROPERTY-NAME,
for e.g. mapred.queue.default.submit-job-acl.
The number of queues configured in this parameter could depend on the
type of scheduler being used, as specified in
mapred.jobtracker.taskScheduler. For example, the JobQueueTaskScheduler
supports only a single queue, which is the default configured here.
Before adding more queues, ensure that the scheduler you've configured
supports multiple queues.
</description>
</property>
修改生效后通过jobtrack界面可以看到配置的队列信息:

要对队列进行控制, 还需要编辑mapred-queue-acls.xml文件
<property>
<name>mapred.queue.queue1.acl-submit-job</name>
<value>' '</value>
<description> Comma separated list of user and group names that are allowed
to submit jobs to the 'default' queue. The user list and the group list
are separated by a blank. For e.g. user1,user2 group1,group2.
If set to the special value '*', it means all users are allowed to
submit jobs. If set to ' '(i.e. space), no user will be allowed to submit
jobs.

It is only used if authorization is enabled in Map/Reduce by setting the
configuration property mapred.acls.enabled to true.
Irrespective of this ACL configuration, the user who started the cluster and
cluster administrators configured via
mapreduce.cluster.administrators can submit jobs.
</description>
</property>
要配置多个队列, 只需要重复添加上面配置信息,修改队列名称和value值,为方便测试,queue1禁止所有用户向其提交作业.
要使该配置生效, 还需要修改mapred-site.xml,将mapred.acls.enabled值设置为true
<property>
<name>mapred.acls.enabled</name>
<value>true</value>
<description> Specifies whether ACLs should be checked
for authorization of users for doing various queue and job level operations.
ACLs are disabled by default. If enabled, access control checks are made by

1 2 3  下一页

Tags:hadoop mapred queue

编辑录入:爽爽 [复制链接] [打 印]
赞助商链接