Apache Accumulo Installation Guide
Contents
1. eee eee eee eee eee where service is one of master monitor gc tracer Or tserver Verify your installation by following the instructions in the Test the Accumulo ShellError Reference source not found section 14 Apache Accumulo Installation Guide Configuring Cloudera Manager for Accumulo 1 6 on CDH 4 Step 7 Optional Configuration for Performance Tuning Relaxing WAL Durability When using the BatchWriter API to ingest data into Accumulo greater write throughput can be attained at the cost of reduced data durability Cloudera recommends using this setting only in environments with reliable UPS To enable this setting add the following text to the accumulo site xm file and distribute the change across hosts running Accumulo roles in the cluster property lt name gt tserver wal sync method lt name gt lt value gt hflush lt value gt lt property gt You should restart all Tablet Servers following this change Configuring Cloudera Manager for Accumulo 1 6 on CDH 4 Cloudera Manager can manage an Accumulo 1 6 service on CDH 4 Follow these setup steps and then continue at Step 3 Configure HDFS in Install Apache Accumulo using Cloudera Manager Step 1 Add the Remote Parcel Repository for Accumulo 1 6 0 cdh4 6 0 Cloudera Manager comes with repository information for running Accumulo 1 6 0 on CDH 5 To run on CDH 4 you will need to add another remote parcel repository2. 2 Navigate to your cluster s Sqoop 1 Client service page Click Configuration Search for sqoop env sh Look for the Gateway Default Group Advanced section s property for Sqoop 1 Client Client Advanced Configuration Snippet Safety Valve for sqoop conf sqoop env sh Click the field and add the snippet appropriate for your installation ensuring that each line ends with a bash comment hash e For parcels export ACCUMULO CONF DIR etc accumulo con f export ACCUMULO HOME opt cloudera parcels ACCUMULO lib accumulo f export HADOOP CLIENT HOME opt cloudera parcels CDH lib hadoop client export HADOOP PREFIX opt cloudera parcels CDH lib hadoop export ZOOKEEPER HOME opt cloudera parcels CDH lib zookeeper e For packages export ACCUMULO CONF DIR etc accumulo con f export HADOOP CLIENT HOME usr lib hadoop client export HADOOP PREFIX usr lib hadoop export ZOOKEEPER HOME usr lib zookeeper 6 Save your changes with a descriptive message such as Sqoop changes for Accumulo 7 Redeploy client configurations for the Sqoop 1 Client service Sqoop 1 under CDH 4 or without Cloudera Manager To use Sqoop integration you must perform the following configuration changes 1 If you do not already have etc sgoop con sgoop env sh create it 22 Apache Accumulo Installation Guide 2 3 4 Using LZO Compression with Accumulo cp etc sqoop conf sqoop env template sh ete sqoop con
3. S se Parcels 2 Under the parcel entry for ACCUMULO 1 6 0 1 cdh5 1 4 p0 116 or later click Download ACCUMULO 1 56 0 1 cdh5 1 4 pO 116 Avallable Remotely Download 3 Underthe cluster you wantto install on for example Cluster 1 find the Accumulo parcel and click Distribute Under the cluster you want to install on for example Cluster 1 find the Accumulo parcel and click Activate You will be prompted to restart the cluster Because the Accumulo parcel was not previously in use you can safely skip this step and click Close Step 3 Configure HDFS Cloudera strongly recommends that you establish an HDFS Name Service on the cluster that will run Accumulo Due to the way Accumulo manages files within HDFS doing so greatly reduces administrative tasks in the future if a NameNode needs to be replaced or moved To set up an HDFS Name Service follow the instructions for enabling HDFS High Availability To guard against data loss you must configure HDFS to durably write data on file close If the following configuration changes are not made Accumulo issues warning messages until the problem is corrected 1 Navigate to your cluster s HDFS service page Apache Accumulo Installation Guide 3 8 9 Install Apache Accumulo using Cloudera Manager Click the Configuration tab Search for hdfs site xml Search for the Service Wide Advanced section s property for HDFS Service Advanced
4. Using Accumulo with Maven If you want to build applications or tools with Cloudera s packaging of Accumulo and you are using Maven or Ivy for dependency management you can pull the Accumulo artifacts from the Cloudera Maven repository The repository is available at https repository cloudera com artifactory cloudera repos The following is a sample snippet from a POM pom xml file lt repositories gt lt repository gt lt id gt cloudera lt id gt lt name gt Cloudera Releases Repository lt name gt lt uri gt https repository cloud era com artifactory cloudera repos lt url gt lt repository gt lt repositories gt CDH 5 Compatible Releases CDH 5 1 Compatible Releases The following table lists the project name groupld artifactld and version required to access each CDH 5 compatible artifact Client applications should only require the accumulo core artifact as a dependency and may need the accumulo maven plugin for running integration tests Project groupld artifactld Accumulo org apache accumulo accumulo 1 6 0 cdh5 1 4 org apache accumulo accumulo core 1 6 0 cdh5 1 4 org apache accumulo accumulo examples 1 6 0 cdh5 1 4 simple 24 Apache Accumulo Installation Guide Using Accumulo with Maven org apache accumulo accumulo fate 1 6 0 cdh5 1 4 org apache accumulo accumulo gc 1 6 0 cdh5 1 4 org apache accumulo accumulo master 1 6 0 cdh5 1 4 org apache accumulo accumulo maven plugi
5. sqgoop env sh Add the following exports to this sgoop env sh file be sure they match your actual installation locations e For parcels export ACCUMULO CONF DIR etc accumulo conf export ACCUMULO HOME opt cloudera parcels ACCUMULO lib accumulo export HADOOP CLIENT HOME opt cloudera parcels CDH lib hadoop client export HADOOP PREFIX opt cloudera parcels CDH lib hadoop export ZOOKEEPER HOME opt cloudera parcels CDH lib zookeeper e For packages export ACCUMULO CONF DIR etc accumulo conf export HADOOP CLIENT HOME usr lib hadoop client export HADOOP PREFIX usr lib hadoop export ZOOKEEPER HOME usr lib zookeeper Save your changes Synchronize this file across all nodes that will run 5 commands Using LZO Compression with Accumulo Optionally you may enable LZO compression for use with Accumulo No special instructions are needed for an installation based on distribution packaging If you are using parcels follow these instructions e l a 10 11 12 13 14 15 Install the GPLExtras Parcel Go to the HDFS service Select the Configuration tab Search for the io compression codecs property In the Compression Codecs entry click in the field and click the to add a new entry Add the following codecs for LZO e com hadoop compression zo LzoCodec e com hadoop compression lzo LzopCodec Save the configuration for the HDFS Service Go to the Accumulo 1 6 service Select the Configuratio
6. 1 Click the Administration menu and select Settings Administration Settings e Alerts Select the Parcels category Edit the configuration item for Remote Parcel Repository URLs Click the next to the last parcel repo to add a new URL Paste the following URL http archive cloudera com accumulo parcels latest 6 Click Save Changes Step 2 Install and Configure Cloudera Manager 5 and CDH 4 6 0 Follow the documentation to install and configure Cloudera Manager 5 with CDH 4 6 0 Be sure to select parcels for cluster components and choose an appropriate CDH 4 version Apache Accumulo Installation Guide 15 Configuring Cloudera Manager for Accumulo 1 6 on CDH 4 1 Besure the Use Parcels option is selected Choose Method Use Packages e 5 Use Parcels Recommended e 2 Select version CDH 4 6 0 1 cdh4 6 0 p0 26 for the CDH parcel Select the version of CDH O COH 5 1 0 1 cdh5 1 0 p0 55 CDH 5 0 3 1 cdh5 0 3 pO 35 O CDH 4 7 0 1 cdh4 7 0 p0 40 CDH 4 6 0 1 cdh4 6 0 pO 26 3 Click Continue and follow the rest of the installation steps as described in the documentation Accumulo requires that you set up HDFS and Zookeeper Other services are optional Step 3 Install the Accumulo Parcel 1 From the Hosts tab select Parcels Hosts Diagnostic 35 ee Parcels 2 Under the ACCUMULO 1 6 0 1 cdh4 6 0 p0 261 parcel click Download ACCUMULO 1 6 0 1 cdh4 6 0 p0 261 Available Re
7. A MK Eo 22 USING LZO COMPRESSION WITH KATAGA 23 USING ACCUMULO WITH 24 DEFAULT PORIS 26 CREATING A LOCAL kok 27 About this Guide About this Guide This guide describes how to install Cloudera s packaging of Apache Accumulo for use with CDH Introducing Apache Accumulo Apache Accumulo is an ideal solution for government agencies looking for a secure distributed NoSQL data store to serve their most performance intensive Big Data applications Accumulo is an open source project integrated with Hadoop and provides the ability to store data in massive tables billions of rows millions of columns for fast random access Accumulo was created and contributed to the Apache Software Foundation by the National Security Agency NSA It has quickly gained adoption as a Hadoop based key value store for applications that have unique and stringent information security requirements Known Issues Known issues with the Accumulo 1 6 service in Cloudera Manager See the known issues document for your release of Cloudera Manager for problems and workarounds specific to running an Accumulo 1 6 service Prerequisites Accumulo depends on HDFS and ZooKeeper libraries and configuration information TabletServers should be collocated with DataNodes Optionally you can us
8. Apache Accumulo from Distribution Packages Click Continue Configure the Accumulo Instance Secret It is strongly recommended that you protect this secret because the security of the Accumulo service relies on it Configure the Accumulo Instance Name Configure the Trace User and Trace Password settings It is strongly recommended that you not leave the Trace User set to root because this is not a secure configuration Click Continue Wait while Cloudera Manager does initial service set up Click Continue Click Finish Verify your installation by following the instructions in the Test the Accumulo Shell section Step 5 Optional Configuration for Performance Tuning Relaxing WAL Durability When using the Bat chWriter API to ingest data into Accumulo you can attain greater write throughput at the cost of reduced data durability Cloudera recommends using this setting only in environments with reliable UPS To enable this setting perform the following configuration changes 1 2 Navigate to your cluster s Accumulo 1 6 service page Click Configuration and then click View and Edit Search for Tablet Server accumulo site xml Find the Tablet Server Default Group Advanced section s property for Tablet Server Advanced Configuration Snippet Safety Valve for accumulo site xml Click the field and add the following snippet lt property gt lt name gt tserver wal sync method lt name gt lt value gt hfl
9. Configuration Snippet Safety Valve for hdfs site xml Click the field and add this snippet lt name gt dfs datanode synconclose lt name gt lt value gt true lt value gt lt property gt Search for any Gateway group properties labeled HDFS Client Advanced Configuration Snippet Safety Valve for hdfs site xml Click the field and add this snippet lt property gt lt name gt dfs datanode synconclose lt name gt lt value gt true lt value gt lt property gt Save your changes with a descriptive message such as HDFS changes for Accumulo Redeploy client configurations for the HDFS service 10 Restart the HDFS service Ila 2 Step 4 Add the Accumulo Service Navigate to the Cloudera Manager Home page Click the actions menu for the cluster you want to add the Accumulo service to for example Cluster 1 and select Add a Service niia ai Select Accumulo 1 6 and click Continue Select the dependent services and click Continue Assign the Accumulo roles to the hosts in your cluster Cloudera recommends assigning a Tablet Server role on each host that is assigned the DataNode role The Monitor Garbage Collector Tracer and Master roles should all be assigned to non DataNodes The Gateway role should be assigned to any hosts where you wantto use Accumulo that do not already have other Accumulo roles assigned 4 Apache Accumulo Installation Guide 10 11 12 13 Install
10. accumulo tserver log LocalWALRecovery Start the Accumulo 1 6 cluster On starting Accumulo will complete the upgrade process This can take some time Check the log files for the Master role for progress Upgrading from CDH 4 to CDH 5 Cloudera does not support running Accumulo 1 4 on CDH 5 If you are running Accumulo 1 4 on CDH 4 follow the instruction above to upgrade to Accumulo 1 6 before upgrading to CDH 5 18 Apache Accumulo Installation Guide Upgrading from CDH 4 to CDH 5 Before upgrading you should plan for downtime Cloudera does not support rolling upgrades from CDH 4 to CDH 5 Additionally you will have to remove and reinstall the Accumulo service as a part of upgrading Step 1 Document and Remove Accumulo 1 6 for CDH 4 1 a E AA AN Stop the Accumulo 1 6 service Go to the Accumulo 1 6 service page Click the Configuration tab Click through the configuration settings and write down any non default settings Click the Instances tab Make a note of your role assignments Return to the Cloudera Manager home page Under actions for the Accumulo 1 6 Service select delete Step 2a Update Parcels and Upgrade CDH If you are using parcels for your cluster follow these instructions to update your CDH version 4 5 Go to the Hosts and then Parcels page You should have an active parcel for Accumulo 1 6 0 cdh4 6 0 You should Deactivate Remove from Hosts and Delete this par
11. with out of sync configurations will not be allowed to join the cluster Step 5 Initialize Accumulo To initialize Accumulo 1 Create the accumulo and user accumulo directories in HDFS and change their ownership to the accumulo user sudo su hdts hadoop fs mkdir accumulo user accumulo hadoop fs chown accumulo supergroup accumulo user accumulo hadoop fs chmod 751 accumulo hadoop fs chmod 750 user accumulo 2 On the Accumulo Master enter the following commands to initialize Accumulo and follow the prompts to name your instance for this example cloudera and set a root password sudo i service accumulo master init util Initialize INFO Hadoop Filesystem is hd sz kocalhost localdomaini8020 util Initialize INFO gt Accumulo data dir is accumulo util Initialize INFO Zookeeper server is localhost 2181 util Initialize INFO Checking if Zookeeper is available If this hangs then you need to make sure zookeeper is running Instance name cloudera Confirm initial password Cor EOOE Tera conf Configuration WARN dfs replication min is deprecated Instead use dfs namenode replication min conf Configuration WARN dfs block size is deprecated Instead use dfs blocksize security ZKAuthenticator INFO Initialized root user with username i l l l l l l l l l l l l l l l l l l l l l l Enter initial password for root l l l l l l l l l l l l
12. CUMULO 1 6 ON CDH 4 sessscessscsssseesscessveseseessscsssstos 15 STEP 1 ADD THE REMOTE PARCEL REPOSITORY FOR 6 5 6 15 STEP 2 INSTALL AND CONFIGURE CLOUDERA MANAGER 5 AND CDH 4 6 15 STEP 3 INSTALL THE ACCUMULO PARCE naasa GANAN AA AA 16 UPGRADING FROM 17 UPGRADING WITH CLOUDERA IMANAGER a 1 UG DING NARE yetetetaes 15 UPGRADING FROM CDH TO CIDIS 18 STEP 1 DOCUMENT AND REMOVE ACCUMULO 1 6 FOR CDH 4 u 0 000000000 0m aa 19 STEP 2A UPDATE PARCELS AND UPGRADE sane KIO NARE KD KR rl KO VA KS SVIN 19 STEP 2B UPDATE PACKAGES AND UPGRADE CD FE asento KKS ED KOLOSE ANAN NGE NGE dab Ende 19 STEP ADD ACCUMULO LJO FOR CDH kva baa o 20 MES ETHIE ACCUMULO SHELL AA AA ANAKAN 20 USING SQOOP T WITH ACCUMULO mGA NAGA va ka 22 SQOOP 1 CLIENT UNDER CDH 5 AND CLOUDERA MANAGER sudar bad a aaa 22 SQOOP 1 UNDER CDH 4 OR WITHOUT CLOUDERA MANAGER sesa o LE A AN E
13. ccumulo accumulo server base 1 6 0 cdh4 6 0 org apache accumulo accumulo start 1 6 0 cdh4 6 0 org apache accumulo accumulo test 1 6 0 cdh4 6 0 org apache accumulo accumulo trace 1 6 0 cdh4 6 0 org apache accumulo accumulo tracer 1 6 0 cdh4 6 0 org apache accumulo accumulo tserver 1 6 0 cdh4 6 0 Default Ports If your cluster is running firewall software you may need to allow communication between hosts on specific ports The following table lists the default port for each server process and the configuration property used to change that value Garbage Collector 50091 gc port client Master 10010 master port client Monitor Log Forwarding 4560 monitor port log4j Monitor Client Port 50091 monitor port client Tablet Server 10011 tserver port client 26 Apache Accumulo Installation Guide Creating a Local yum Repository Creating a Local yum Repository This section explains how to set up a local yum repository that you can then use to install Accumulo on the machines in your cluster There are a number of reasons you might want to do this for example e The computers in your cluster may not have Internet access You can still use yum to do an installation on those machines by creating a local yum repository e You may want to keep a stable local repository to ensure that any new installations or re installations on existing cluster members use exactly the same bits e Using a local repository may be the most eff
14. ccumulo monitor and so on After the package installation return to Step 3 Configure HDFS below to continue installing the Accumulo 1 6 service Note Managing a CDH 4 cluster The instructions in this section work with CDH 5 You may also use Cloudera Manager to manage the Accumulo 1 6 service on a CDH 4 cluster To do so follow the instructions in Configuring Cloudera Manager for Accumulo 1 6 on CDH 4 and then return to Step 3 Configure HDFS below to continue installing the Accumulo 1 6 service Step 1 Install and Configure Cloudera Manager and CDH Follow the documentation to install and configure Cloudera Manager 5 with CDH During the installation you choose compatible CDH and Accumulo parcels while following these instructions 1 Besure the Use Parcels option is checked Choose Method Use Packages 5 Use Parcels Recommended e 2 Select version CDH 5 1 0 1 cdh5 1 0 p0 55 or higher for the CDH parcel Select the version of CDH CDH 5 1 0 1 cdh5 1 0 p0 55 2 GDH 5 0 3 1 cdh5 0 3 p0 35 3 CDH 4 7 0 1 cdhn4 7 0 p0 40 2 GDH 4 6 0 1 cdh4 6 0 p0 26 2 Apache Accumulo Installation Guide Install Apache Accumulo using Cloudera Manager 3 Click Continue and follow the rest of the installation steps as described in the documentation Accumulo requires that you set up HDFS and Zookeeper Other services are optional Step 2 Install the Accumulo Parcel 1 From the Hosts tab select Parcels 7 Hosts Diagnostic
15. cel After deactivating CM will prompt you to restart services Close this restart prompt without restarting You should see CDH version cdh5 1 0 in the list of downloadable parcels DOWNLOAD and DISTRIBUTE this parcel CDH 5 1 0 1 cdh5 1 0 p0 55 Available Remotely Download Once the CDH 5 1 0 parcel is distributed click the Upgrade button and follow the upgrade wizard according to the documentation for parcels Follow the CDH 5 instructions found in Step 2 Install the Accumulo Parcel above Step 2b Update Packages and Upgrade CDH If you are using packages for your cluster follow these instructions to update your CDH version Apache Accumulo Installation Guide 19 Test the Accumulo Shell 1 Remove the Accumulo packages from all nodes in your cluster using your system s package manager Remove the Accumulo repo from your package manager on all nodes in the cluster 3 Upgrade CDH according to the documentation for packages Follow the CDH 5 instructions found in Install Apache Accumulo from Distribution Packages steps 1 and 2 above Step 3 Add Accumulo 1 6 for CDH 5 1 Add the Accumulo 1 6 service to your cluster 2 Assign roles to match those you wrote down in Step 1 3 Inthe initial configuration be sure the instance name and secret match those you wrote down in Step 1 4 The wizard will try to initialize Accumulo and fail This is expected because you already have an Accumulo instance de
16. cloudera Ask Bigger Questions Apache Accumulo Installation Guide for using Cloudera s packaging of Accumulo for CDH Important Notice 2010 2015 Cloudera Inc All rights reserved Cloudera the Cloudera logo Cloudera Impala Impala and any other product or service names or slogans contained in this document except as otherwise disclaimed are trademarks of Cloudera and its suppliers or licensors and may not be copied imitated or used in whole or in part without the prior written permission of Cloudera or the applicable trademark holder Hadoop and the Hadoop elephant logo are trademarks of the Apache Software Foundation Apache Accumulo Accumulo Apache the Apache feather logo and the Apache Accumulo project logo are trademarks of the Apache Software Foundation All other trademarks registered trademarks product names and company names or logos mentioned in this document are the property of their respective owners Reference to any products services processes or other information by trade name trademark manufacturer supplier or otherwise does not constitute or imply endorsement sponsorship or recommendation thereof by us Complying with all applicable copyright laws is the responsibility of the user Without limiting the rights under copyright no part of this document may be reproduced stored in or introduced into a retrieval system or transmitted in any form or by any means electronic mechanical photocopying rec
17. e Logger WAL Directories and the maximum heap sizes for both the Tablet Server and Logger roles Install the parcel for Accumulo 1 6 0 cdh4 6 0 according to Step 3 Install the Accumulo Parcel in the section on Configuring Cloudera Manager for Accumulo 1 6 on CDH 4 Add Accumulo 1 6 service to your cluster In the role assignments page you should have the same deployment as you had with 1 4 Specifically the tablet servers should match exactly what they were in Accumulo 1 4 In the Review Changes page in the wizard set the properties to have the same values that they were in Accumulo 1 4 The Accumulo Instance Secret and Accumulo Instance Name must match exactly what they were in Accumulo 1 4 The wizard will try to initialize Accumulo and fail This is expected because you already have an Accumulo instance defined Cancel the wizard Browse to the Accumulo 1 6 service page Click on Configuration tab Search for logger Click the entry for Logger WAL Directories and enter the same value that it was in Accumulo 1 4 Search for tserver max Click the value for Tablet Server Max Heapsize Make sure itis setto the sum of the Accumulo 1 4 Tablet Server and Logger maximum heap sizes Apache Accumulo Installation Guide 17 16 17 18 19 20 21 Upgrading from CDH 4 to CDH 5 Enter a descriptive name for your changes such as Accumulo upgrade settings and click Save Changes Run Upgrade Accumu
18. e Accumulo with MapReduce and Sqoop 1 Cloudera recommends that MapReduce users rely on the stand alone service in CDH 4 and on YARN includes MapReduce 2 in CDH 5 The current release of Cloudera s packaging of Apache Accumulo is tested for use with CDH 4 6 0 and higher and CDH 5 1 0 and higher Cloudera Manager has been tested for managing this release with both parcels and package RPM DEB installations with CDH 5 1 0 and higher For full cluster installations Cloudera strongly recommends following the Tips and Guidelines in the CDH 5 Installation Guide Apache Accumulo Installation Guide 1 Install Apache Accumulo using Cloudera Manager Install Apache Accumulo using Cloudera Manager This section describes how to install Cloudera s packaging of Accumulo by using Cloudera Manager 5 1 0 or higher If you prefer to install Accumulo from packages and manage the cluster manually skip this section and see the Install Apache Accumulo from Distribution Packages section on page 5 Note Managing a cluster installed with packages The instructions in this section use parcels You can also use Cloudera Manager to manage the Accumulo 1 6 service when installing packages To do so follow Steps 1 and 2 in Install Apache Accumulo from Distribution Packages During the package installation sub step of Step 2 follow the instructions for All client hosts only In particular do not install any of the role specific packages such as accumulo master a
19. e lt name gt lt value gt true lt value gt lt property gt 2 Synchronize the updated hdfs site xml file across your cluster 3 Restart all HDFS DataNodes Step 4 Configure Accumulo for Your Environment After installation follow the steps in this section to configure Accumulo for your environment 1 Onevery host configure the following properties in etc accumulo conf accumulo site xml with the proper values for your environment PEODEEEV lt name gt instance zookeeper host lt name gt lt value gt localhost 2181 lt value gt lt description gt comma separated list of zookeeper servers lt description gt lt property gt S PEODSEEVo lt name gt instance secret lt name gt lt value gt DEFAULT lt value gt lt description gt A secret unique to a given instance that all servers must know in order to communicate with one another Change it before initialization To change it later use bin accumulo org apache accumulo server util ChangeSecret oldpasswd newpasswd and then update this file lt desCriprion lt property gt property lt name gt tserver memory maps max lt name gt lt value gt 256M lt value gt Apache Accumulo Installation Guide 11 Install Apache Accumulo from Distribution Packages property lt name gt tserver cache data size lt name gt lt value gt 15M lt value gt lt property gt property lt name gt tserver cache index size lt name gt lt val
20. every DataNode 1 Depending on your needs install and deploy either CDH 5 or CDH 4 Follow instructions under the documentation for CDH 5 Installation or CDH 4 Installation as appropriate 2 Install and deploy ZooKeeper Follow the instructions for CDH 5 or CDH4 as appropriate Install each type of daemon package on the appropriate systems s as follows Where to install Install commands Accumulo Master host running Red Hat CentOS compatible sudo yum clean all sudo yum install accumulo master sudo zypper clean all sudo zypper install accumulo master Ubuntu or Debian sudo apt get update sudo apt get install accumulo master Accumulo Monitor host running Red Hat CentOS compatible sudo yum clean all sudo yum install accumulo monitor sudo zypper clean all sudo zypper install accumulo monitor Ubuntu or Debian sudo apt get update sudo apt get install accumulo monitor Apache Accumulo Installation Guide 9 Install Apache Accumulo from Distribution Packages Accumulo Garbage Collector host running Red Hat CentOS compatible sudo yum clean all sudo yum install accumulo gc sudo zypper clean all sudo zypper install accumulo gc Ubuntu or Debian sudo apt get update sudo apt get install accumulo gc Accumulo Tracer host running Red Hat CentOS compatible sudo yum clean all sudo yum install accumulo tracer sudo zypper clean all sudo zypper install accumulo tracer Ubuntu or Debian sudo a
21. fined Cancel the wizard Go to the Accumulo 1 6 Service page Click the Configuration tab Update the service configuration to match what you wrote down in Step 1 E ON NO Run Deploy Client Configuration from the Actions menu for the cluster on the CM Home page 9 Start the Accumulo 1 6 service Test the Accumulo Shell You can now run the Accumulo shell on any client hosts for CM installs these are hosts assigned the Gateway role in your cluster By default the user root is created and given the password secret If you did not set a different password during install Cloudera strongly recommends that you change the root user password The following steps will verify that the Accumulo shell works while allowing you to change the root user password 1 Launch the Accumulo shell for the default root user gt accumula shell u root Enter current password for toot accumulo Shell Apache Accumulo Interactive Shell version 1 6 0 cdh4 6 0 instance name accumulo instance 98sboditil eozji 46 71 9e81E 698917 0dGo5E 20 Apache Accumulo Installation Guide Test the Accumulo Shell root accumulo gt passwd Enter current password Tor root Enter new password Please confirm new password for root xxxxxxxxx FOOT accumulo root accumulo gt exit o accumulo shell u root Enter current password tor root U accumulo s Shell Apac
22. he Accumulo Interactive Shell version 1 6 0 cdh4 6 0 instance name accumulo instance id 986311f11 c323 4611 9281f2 69857f210d635f type help for a list of available commands root Gaccumulo gt root accumulo gt tables IMETADATA trace root accumulo gt If the trace table does not exist make sure that you have created the trace user Use the same password you used for the trace password setting in etc accumulo conf accumulo site xml for a manually managed cluster or the Trace Password setting in Cloudera Manager installations root cloudera gt createuser trace Enter new password for trace x x Please confirm new password for trace x root cloudera gt grant System CREATE TABLE s u trace root cloudera gt tables METADATA Apache Accumulo Installation Guide 21 Using Sqoop 1 with Accumulo For more information on using the Accumulo shell see the Accumulo user manual Using Sqoop 1 with Accumulo CDH 4 6 0 and higher and CDH 5 1 0 and higher include Sqoop bindings for import export of data with Accumulo For instructions on invoking Sqoop with Accumulo as a source or sink see the Sgoop documentation When running the sqoop command you may see warning messages about failing to create usr lib accumulo logs These messages are safe to ignore Sqoop 1 Client under CDH 5 and Cloudera Manager To use Sqoop integration you must perform the following configuration changes 1
23. icient way to distribute the software to cluster members To set up your own internal mirror do the following Note Before You Start These instructions assume you already have the appropriate Cloudera repo file on the system on which you are going to download the local repository If this is not the case follow the instructions in Adding the Accumulo 1 Ona computer that does have Internet access install the yum uti1s and createrepo packages if they are not already installed yum utils includes the reposync command 2 Onthe same computer as the previous step download the yum repository to a temporary location On Red Hat CentOS 6 you can use a command such as Note cloudera accumulo is the name of the repository on your system the name is in square brackets and usually is on the first line of the repo file which in this example is ke e yum BSpos cloudera accumuloe repo 3 Copy all of the RPMs to the machine that will serve the local repository and place them in a directory served by your web server For this example it is called var www html accumulo 1 6 0 RPMS x86 64 or 1386 for 32 bit systems Make sure Apache Accumulo Installation Guide 27 Creating a Local yum Repository you can remotely access the files in the directory you just created the URL should look like http lt yourwebserver gt accumulo 1 6 0 RPMS 4 Onthe server in step 3 go to var www html accumulo 1 6 0 and type the fo
24. l l l l root at the request of user SYSTEM l l Warnings You will be warned here if you did not change your instance secret in Apache Accumulo Installation Guide 13 Install Apache Accumulo from Distribution Packages ise ace unmnu Comm acecumullo Hire xml Ifthe Hadoop Filesystem is line contains file or s instead of hdfs HDFS is not properly configured Step 6 Start Accumulo To start Accumulo 1 Run the following commands on the following hosts For the following service Run this command Accumulo Master sudo 1 service accumulo master Start Accumulo Monitor sudo 1 service accumulo monitor Start Accumulo Garbage Collector sudo 1 service accumulo gc start Accumulo Tracer sudo 1 service accumulo tracer Start All cluster hosts except Accumulo Master sudo 1 service accumulo tserver Accumulo Monitor Accumulo Garbage start Collector and Accumulo Tracer hosts 2 Connect to Accumulo on http localhost 50095 You can check the status of each daemon with the following command where service is one of master monitor gc tracer Or tserver 3 You can stop each daemon with the following command OEO ee eee
25. llowing command This will create or update the necessary metadata so yum can understand this new repository you will see a new directory named repodata Important Check the permissions of the subdirectories under var www html accumulo 1 6 0 Make sure they are all readable by your web server user 5 Edit the repo file you downloaded previously and replace the line starting with baseurl or mirrorlist with baseurl http lt yourwebserver gt accumulo 1 6 0 6 Save this modified repo file in etc yum repos d and check that you can install Accumulo through yum Example Once you have confirmed that your internal mirror works you can distribute this modified repo file to all your machines and they should all be able to install Accumulo without needing access to the Internet Follow the instructions in Step 2 Install Accumulo 28 Apache Accumulo Installation Guide
26. lo from the Actions menu on Accumulo 1 6 service page Start the Accumulo 1 6 service On starting Accumulo will complete the upgrade process This can take some time Check the log files for the Master role for progress Run Deploy Client Configuration from the Actions menu on Accumulo 1 6 service page Once you have checked that Accumulo 1 6 is working properly delete the Accumulo 1 4 service from your cluster Upgrading via RPMs If you are using Cloudera Manager to handle a package based installation follow the instructions above substituting the Accumulo yum repository and package update steps in this section for the changes to parcels If you are manually managing a package based installation follow all of these instructions 1 Verify that there are no outstanding FATE operations under Accumulo 1 4 accumulo org apache accumulo server fate Admin print Stop the Accumulo 1 4 cluster On every host replace the yum repository for Accumulo according to the appropriate CDH 4 entry for your OS as seenin Step 1 Add or Build the Accumulo Repository from the section Install Apache Accumulo from Distribution Packages As root on every host upgrade the Accumulo packages sudo yum clean all sudo yum update accumulo On every host that was running the Tablet Server or Logger roles run a utility to rewrite Accumulo 1 4 WALs to HDFS in a format that Accumulo 1 6 can read sudo u accumulo accumulo N org apache
27. motely Download 3 Under the cluster you want to install on for example Cluster 1 find the Accumulo parcel and click Distribute 4 Under the cluster you want to install on for example Cluster 1 find the Accumulo parcel and click Activate 16 Apache Accumulo Installation Guide De Upgrading from Accumulo 1 4 You will be prompted to restart the cluster Because the Accumulo parcel was not previously in use you can safely skip this step and click Close Now continue with the instructions in Step 3 Configure HDFS Install Apache Accumulo using Cloudera Manager Upgrading from Accumulo 1 4 If you have a cluster currently running Cloudera s packaging for Accumulo 1 4 on CDH 4 the following instructions will guide you through upgrading to Cloudera s packaging for Accumulo 1 6 on CDH 4 You should plan for down time the Accumulo service must be stopped as a part of the upgrade Upgrading with Cloudera Manager lu 10 11 12 13 14 15 Upgrade your Cloudera Manager to 5 1 0 or later using the instructions for updating to the latest Cloudera Manager version Follow the instructions in Step 1 Add the Remote Parcel Repository for Accumulo 1 6 0 cdh4 6 0 in the section on Configuring Cloudera Manager for Accumulo 1 6 on CDH 4 Stop Accumulo 1 4 service in Cloudera Manager Take note of your configuration settings In particular you will need the Accumulo Instance Secret Accumulo Instance Nam
28. n 1 6 0 cdh5 1 4 org apache accumulo accumulo minicluster 1 6 0 cdh5 1 4 org apache accumulo accumulo monitor 1 6 0 cdh5 1 4 org apache accumulo accumulo proxy 1 6 0 cdh5 1 4 org apache accumulo accumulo server base 1 6 0 cdh5 1 4 org apache accumulo accumulo start 1 6 0 cdh5 1 4 org apache accumulo accumulo test 1 6 0 cdh5 1 4 org apache accumulo accumulo trace 1 6 0 cdh5 1 4 org apache accumulo accumulo tracer 1 6 0 cdh5 1 4 org apache accumulo accumulo tserver 1 6 0 cdh5 1 4 CDH 4 Compatible Releases CDH 4 6 Compatible Releases The following table lists the project name groupld artifactld and version required to access each CDH 4 compatible artifact Client applications should only require the accumulo core artifact as a dependency and may need the accumulo maven plugin for running integration tests Project groupld artifactld vesion Accumulo org apache accumulo accumulo 1 6 0 cdh4 6 0 org apache accumulo accumulo core 1 6 0 cdh4 6 0 org apache accumulo accumulo examples 1 6 0 cdh4 6 0 simple org apache accumulo accumulo fate 1 6 0 cdh4 6 0 Apache Accumulo Installation Guide 25 Default Ports org apache accumulo accumulo gc 1 6 0 cdh4 6 0 org apache accumulo accumulo master 1 6 0 cdh4 6 0 org apache accumulo accumulo maven plugin 1 6 0 cdh4 6 0 org apache accumulo accumulo minicluster 1 6 0 cdh4 6 0 org apache accumulo accumulo monitor 1 6 0 cdh4 6 0 org apache accumulo accumulo proxy 1 6 0 cdh4 6 0 org apache a
29. n tab Search for the accumulo classpath property In the Additional Classpath entry click in the field if needed click to add a new entry Add the following path that corresponds to the version you are using 1 6 0 cdh4 6 0 opt cloudera parcels HADOOP LZO lib hadoop lib hadoop Izo jar 1 6 0 cdh5 1 0 opt cloudera parcels GPLEXTRAS lib hadoop lib hadoop Izo jar Search for service environment In the Service Wide Advanced section s Accumulo 1 6 Service Environment Advanced Configuration Snippet Safety Valve click the field Add the following line that corresponds to the version you are using 1 6 0 cdh4 6 0 LD LIBRARY PATH SLD LIBRARY PATH opt cloudera parcels HADOOP LZO lib hadoop lib native Apache Accumulo Installation Guide 23 16 17 18 19 20 21 Using Accumulo with Maven 1 6 0 cdh5 1 0 LD LIBRARY PATH SLD LIBRARY PATH opt cloudera parcels GPLEXTRAS lib hadoop lib native Search for the accumulo java opts property In the Additional Java Options entry click the field Add to the end of the existing options Dio compression codec lzo class com hadoop compression lzo LzoCodec Save the Accumulo 1 6 service with a description such as Accumulo LZO Redeploy client configurations for the cluster Restart the Accumulo 1 6 services If you have configured Cloudera Manager to install parcels in a location other than opt cloudera parcels change the above paths to reflect that location
30. ording or otherwise or for any purpose without the express written permission of Cloudera Cloudera may have patents patent applications trademarks copyrights or other intellectual property rights covering subject matter in this document Except as expressly provided in any written license agreement from Cloudera the furnishing of this document does not give you any license to these patents trademarks copyrights or other intellectual property The information in this document is subject to change without notice Cloudera shall not be liable for any damages resulting from technical errors or omissions which may be present in this document or from use of this document Cloudera Inc 1001 Page Mill Road Palo Alto CA 94304 1008 info cloudera com US 1 888 789 1488 Intl 1 650 362 0488 www cloudera com Release Information Version 1 6 0 cdh4 6 0 Date July 17 2014 Version 1 6 0 cdh5 1 4 Date March 19 2014 Table of Contents ABOUT THIS GUIDE 1 INTRODUCING APACHE od Dj abne 1 Messo AA 1 Known issues with the Accumulo 1 6 service in Cloudera Manager ss ssssssscssssssssscccsssssss 1 PRET EOIS PAA 1 INSTALL APACHE ACCUMULO USING CLOUDERA MANAGER s ssesssesssse
31. pt get update sudo apt get install accumulo tracer All cluster hosts except Accumulo Master Accumulo Monitor Accumulo Garbage Collector and Accumulo Tracer hosts running Red Hat CentOS compatible sudo yum clean all sudo yum install accumulo tserver sudo zypper clean all sudo zypper install accumulo tserver Ubuntu or Debian sudo apt get update sudo apt get install accumulo tserver All client hosts running Red Hat CentOS compatible sudo yum clean all sudo yum install accumulo SLES sudo zypper clean all sudo zypper install accumulo Ubuntu or Debian sudo apt get update sudo apt get install accumulo 10 Apache Accumulo Installation Guide Install Apache Accumulo from Distribution Packages Step 3 Configure HDFS Cloudera strongly recommends that you establish an HDFS Name Service on the cluster that will run Accumulo Due to the way Accumulo manages files within HDFS doing so greatly reduces administrative tasks in the future if a NameNode needs to be replaced or moved To set up an HDFS Name Service follow the instructions for enabling HDFS High Availability To guard against data loss you must configure HDFS to durably write data on file close If the following configuration changes are not made Accumulo issues warning messages until the problem is corrected 1 Edit the hdfs site xml used in your cluster and ensure it contains the following snippet cproperty lt name gt dfs datanode synconclos
32. recise amd64 cdh or ubuntu trusty amd64 cdh CDH5 only and RELEASE is the name of your distribution which you can find by running Isb release C For example to install Accumulo on top of CDH 5 for 64 bit Ubuntu Lucid deb arch amd64 http archive cloudera com accumulo c5 ubuntu lucid amd64 cdh lucid cdh5 contrib deb src http archive cloudera com accumulo e5 ubuntu lueid amd64 edh lucid cdh5 contrib Building a Debian Repository If you want to create your own apt repository create a mirror of the Accumulo Debian directory and then create an apt repository from the mirror Now continue with Step 2 Install Accumulo 8 Apache Accumulo Installation Guide Install Apache Accumulo from Distribution Packages Step 2 Install Accumulo Important Before proceeding you need to decide where to deploy the Accumulo Master Accumulo Monitor Accumulo Garbage Collector and Accumulo Tracer daemons As a general rule e The Accumulo Master and Accumulo Monitor run on the same master host unless the cluster is large more than a few tens of nodes and the master host or hosts should not run the Accumulo TabletServer service In a large cluster it is especially important that the Accumulo Garbage Collector and Accumulo Tracer run on machines separate from the Accumulo Master Each node in the cluster except the master host s should run the Accumulo TabletServer service In particular these services should be run on
33. rsion Follow this Link Red Hat CentOS Oracle 5 CDH4 CDH 4 for Red Hat CentOS Oracle 5 CDH 5 CDH 5 for Red Hat CentOS Oracle 5 Red Hat CentOS Oracle 6 CDH 4 CDH 4 for Red Hat CentOS Oracle 6 CDH 5 CDH 5 for Red Hat CentOS Oracle 6 Now continue with Step 2 Install Accumulo Building a yum Repository If you want to create your own yum repository download the appropriate repo file create the repo distribute the repo file and set up a web server as described under Creating a Local yum Repository On SLES Systems Use one of the following methods to download the Accumulo repository or packages on SLES systems by using the instructions in one of the following sections e Adding the Accumulo e Building a SLES 6 Apache Accumulo Installation Guide Install Apache Accumulo from Distribution Packages Do this on all systems in the cluster Adding the Accumulo Repository 1 Run the command that corresponds to your CDH version sudo zypper addrepo f http archive cloudera com accumulo c5 sles 11 x86 64 cdh cloudera accumulo repo S sudo zypper addrepo f http archive cloudera com accumulo sles 11 x86 64 cdh cloudera accumulo repo Building a SLES Repository If you want to create your own SLES repository create a mirror of the Accumulo SLES directory by following these instructions that explain how to create a SLES repository from the mirror Now continue with Step 2 Install Accumulo On Ubuntu or Debian System
34. s Use one of the following methods to add or build the Accumulo repository or download the packages on Ubuntu or Debian systems by using the instructions in one of the following sections e Adding the Accumulo Repository e Building a Debian Do this on all the systems in the cluster Adding the Accumulo Repository Create a new file etc apt sources list d cloudera accumulo list with the following contents e CDH 5 For Ubuntu systems deb arch amd64 http archive cloudera com accumulo c5 OS release arch RELEASE CUN5 contrib l deb src http archive cloudera com accumulo c5 OS release arch Apache Accumulo Installation Guide 7 Install Apache Accumulo from Distribution Packages deb arch amd64 http archive cloudera com accumulo OS release arch gt lt RELEASE gt cdh4 Contrib deb src http archive cloudera com accumulo OS release arch lt RELEASE gt cdh4 contrib deb http archive cloudera com accumulo c5 OS release arch RELEASE Cdh5 contrib deb src http archive cloudera com accumulo c5 OS release arch lt RELEASE gt cdhS Contrib deb http archive cloudera com accumulo OS release arch lt RELEASE gt cdh4 contrib deb src http archive cloudera com accumulo OS release arch RELEASE cdh4 contrib where OS release arch is debian squeeze amd64 cdh CDH4 only debian wheezy amd64 cdh CDH5 only ubuntu lucid amd64 cdh ubuntu p
35. ssssesssqesssvesscesussesasseseseesssesesesasos 2 STEP 1 INSTALL AND CONFIGURE CLOUDERA MANAGER AND CDH 10 1 2 STEPZ INSTAECTREXCCUMUCO FARCE ABA 3 SITE Se ge ugo AA 3 STEPASADD IHEACCUNKULO 4 STEP 5 OPTIONAL CONFIGURATION FOR PERFORMANCE TUNING RELAXING WAL DURABILITY 000000000 00oras nnaman 5 INSTALL APACHE ACCUMULO FROM DISTRIBUTION PACKAGES s sssssssssssssssssssssssesssssssqeseessusssqeqsescssssesese 5 STEP 1 ADD OR BUILD THE ACCUMULO REPOSITORY 2 az sts kan sko NANA kaoso abato sA AA AA 6 On Red Har compal Dle SyS TENE si aar Era ANG KES NPO A EE KSA KN SV KKE an kreado 6 Ey secre rarer 6 On kela Debian etna ound AEREA AETIA NAANA EREA 7 SIPAIN TAU MULO AA Se bonpia AA 9 STEP 3 CON FIGURE HDF aa indinin 11 STEP 4 CONFIGURE ACCUMULO FOR YOUR ENVIRONMENT kol 11 STEROSINTIALZE ACCUNUD aa NG 13 ap Kel VIV LOPE AA eee 14 STEP 7 OPTIONAL CONFIGURATION FOR PERFORMANCE TUNING RELAXING WAL DURABILITY ccssscccsecccesecceeneeceeeeenaeeeeeues 15 CONFIGURING CLOUDERA MANAGER FOR AC
36. ue gt 40M lt value gt lt property gt lt property gt lt name gt trace password lt name gt lt value gt trace lt value gt property property lt name gt trace user lt name gt lt value gt trace lt value gt lt property gt 2 Review the configured values For example verify that you changed the value for instance secret 3 Reviewthe service specific options such as Java heap size in the etc default accumulo file ACCUMULO TSERVER OPTS Xmxlg Xmslg XX NewSize 500m XX MaxNewSize 500m ACCUMULO MASTER OPTS Xmx2g Xmslg ACCUMULO MONITOR OPTS Xmx29g Xms256m ACCUMULO GC OPTS Xmx256m Xms256m ACCUMULO GENERAL OPTS XX rUseConcMarkSweepGC XX CMSInitiatingOccupancyFraction 7 5 ACCUMULO OTHER OPTS xmxlg xms256m Important On a multi host cluster replace localhost with the fully qualified domain name FQDN or IP address of the Accumulo Master in the masters monitor gc and tracers files in etc accumulo conf and add the FODN or IP address of the TabletServers one per line to the etc accumulo conf slaves file 12 Apache Accumulo Installation Guide Install Apache Accumulo from Distribution Packages Important On a multi host cluster the contents of the etc accumulo conf directory must always be synchronized across all Accumulo servers within a cluster This can be done using configuration management version control or via a utility such as rsync Servers
37. ush lt value gt lt property gt Save your changes with a descriptive message such as Accumulo WAL Durability Changes Redeploy client configurations Restart the Tablet Servers in your Accumulo 1 6 Service Install Apache Accumulo from Distribution Packages This section describes how to install Cloudera s packaging of Accumulo from packages RPM or DEB instead of using Cloudera Manager Apache Accumulo Installation Guide 5 Install Apache Accumulo from Distribution Packages Step 1 Add or Build the Accumulo Repository e lf you are installing Accumulo on a Red Hat system you can download the Cloudera packages using yum or your web browser e lf you are installing Accumulo on a SLES system you can download the Cloudera packages using zypper YaST or your web browser e Ifyou are installing Accumulo on an Ubuntu or Debian system you can download the Cloudera packages using apt or your web browser On Red Hat compatible Systems Use one of the following methods to add or build the Accumulo repository or download the packages on Red Hat compatible systems by using the instructions in one of the following sections e Adding the Accumulo e Building a yum Do this on all systems in the cluster Adding the Accumulo Repository Follow the link in the table below that matches your Red Hat or CentOS system navigate to the repo file for your system and save itin the etc yum repos d directory For OS Version CDH Ve
Download Pdf Manuals
Related Search