본문 바로가기
Study/Bigdata

인트라넷(폐쇄망) 환경에서 Ambari, HDP 배포하기

redyuika 2019. 5. 6. 18:09

최근, 폐쇄망 환경에서 Ambari,HDP를 배포할 일이 있었습니다. 

조만간 또할것 같지만,  사실 예전에 몇번 해보긴 했는데 오랜만에 하느라 기억이 버벅였네요

 

환경은 다음과 같습니다

centos 7 64bit Server들

 

기본적으로 인터넷환경에서는 Ambari Server 설치후 Ambari Agent가 설치되면서 외부 HDP 레포에서 YUM으로 다운로드 하기 때문에 인터넷이 필요합니다.

 

그럼 폐쇄망 환경은 어떻게 하느냐, 아마 당연하겠지만 , 내부 레포를 만드시면 됩니다

 

필요한 내부레포

1. Centos Base Repo

2. Ambari Repo

3. HDP Repo

 

예전에는 Base레포 없이 했던것 같긴한데, Base레포가 필요한 이유는 Ambari Backend DB를 postgres나 별도의 DB없이 Hive Metastore를 설치할때 Mariadb 또는 MySQL을 필요로 하기 때문입니다.

 

이 외에애도 Ambari가 HDP 레포를 배포하면서 redhat-lsb, python-devel 그리고.. 몇가지 패키지를 기본으로 필요로 하기때문에 Centos Base Repo를 준비해야합니다 

 

Repo를 구축하는 방법은 간단히 구글에서 검색하면 나오기때문에 이부분은 생략하겠습니다

 

일단 HDP3 버전을 기준으로 하자면  (2019년 5월 6일 기준, 최신버전으로 ) 다음과 같습니다

 

 

------------------------------------------------------------------------------------------레포정보

 

Ambari 레포는 다음과 같이 받을수 있고

https://docs.hortonworks.com/HDPDocuments/Ambari-2.7.3.0/bk_ambari-installation/content/ambari_repositories.html

 

 

 

Ambari Repositories

If you do not have Internet access, use the link appropriate for your OS family to download a tarball that contains the software for setting up Ambari.

If you have temporary Internet access, use the link appropriate for your OS family to download a repository file that contains the software for setting up Ambari.

Ambari 2.7.3 Repositories

OS Format URL

RedHat 7

CentOS 7

Oracle Linux 7

 Base URL http://public-repo-1.hortonworks.com/ambari/centos7/2.x/updates/2.7.3.0
Repo File http://public-repo-1.hortonworks.com/ambari/centos7/2.x/updates/2.7.3.0/ambari.repo
Tarball md5 |asc http://public-repo-1.hortonworks.com/ambari/centos7/2.x/updates/2.7.3.0/ambari-2.7.3.0-centos7.tar.gz

amazonlinux 2

 Base URL http://public-repo-1.hortonworks.com/ambari/amazonlinux2/2.x/updates/2.7.3.0
Repo File http://public-repo-1.hortonworks.com/ambari/amazonlinux2/2.x/updates/2.7.3.0/ambari.repo
Tarball md5 |asc http://public-repo-1.hortonworks.com/ambari/amazonlinux2/2.x/updates/2.7.3.0/ambari-2.7.3.0-amazonlinux2.tar.gz
SLES 12 Base URL http://public-repo-1.hortonworks.com/ambari/sles12/2.x/updates/2.7.3.0
Repo File http://public-repo-1.hortonworks.com/ambari/sles12/2.x/updates/2.7.3.0/ambari.repo
Tarball md5 |asc http://public-repo-1.hortonworks.com/ambari/sles12/2.x/updates/2.7.3.0/ambari-2.7.3.0-sles12.tar.gz
Ubuntu 14 Base URL http://public-repo-1.hortonworks.com/ambari/ubuntu14/2.x/updates/2.7.3.0
Repo File http://public-repo-1.hortonworks.com/ambari/ubuntu14/2.x/updates/2.7.3.0/ambari.list
Tarball md5 |asc http://public-repo-1.hortonworks.com/ambari/ubuntu14/2.x/updates/2.7.3.0/ambari-2.7.3.0-ubuntu14.tar.gz
Ubuntu 16 Base URL http://public-repo-1.hortonworks.com/ambari/ubuntu16/2.x/updates/2.7.3.0
Repo File http://public-repo-1.hortonworks.com/ambari/ubuntu16/2.x/updates/2.7.3.0/ambari.list
Tarball md5 |asc http://public-repo-1.hortonworks.com/ambari/ubuntu16/2.x/updates/2.7.3.0/ambari-2.7.3.0-ubuntu16.tar.gz
Ubuntu 18 Base URL http://public-repo-1.hortonworks.com/ambari/ubuntu18/2.x/updates/2.7.3.0
Repo File http://public-repo-1.hortonworks.com/ambari/ubuntu18/2.x/updates/2.7.3.0/ambari.list
Tarball md5 |asc http://public-repo-1.hortonworks.com/ambari/ubuntu18/2.x/updates/2.7.3.0/ambari-2.7.3.0-ubuntu18.tar.gz
Debian 9 Base URL http://public-repo-1.hortonworks.com/ambari/debian9/2.x/updates/2.7.3.0
Repo File http://public-repo-1.hortonworks.com/ambari/debian9/2.x/updates/2.7.3.0/ambari.list
Tarball md5 |asc http://public-repo-1.hortonworks.com/ambari/debian9/2.x/updates/2.7.3.0/ambari-2.7.3.0-debian9.tar.gz

HDP 레포는 다음과 같습니다 

https://docs.hortonworks.com/HDPDocuments/Ambari-2.7.3.0/bk_ambari-installation/content/hdp_31_repositories.html

 

HDP 3.1.0 Repositories - Hortonworks Data Platform

 

docs.hortonworks.com

 

 

HDP 3.1.0 Repositories

OS Version Number Repository Name Format URL

RedHat 7

CentOS 7

Oracle Linux 7

 HDP-3.1.0.0 HDP Version Definition File (VDF) http://public-repo-1.hortonworks.com/HDP/centos7/3.x/updates/3.1.0.0/HDP-3.1.0.0-78.xml
Base URL http://public-repo-1.hortonworks.com/HDP/centos7/3.x/updates/3.1.0.0
Repo File http://public-repo-1.hortonworks.com/HDP/centos7/3.x/updates/3.1.0.0/hdp.repo
Tarball md5 | asc http://public-repo-1.hortonworks.com/HDP/centos7/3.x/updates/3.1.0.0/HDP-3.1.0.0-centos7-rpm.tar.gz
HDP-UTILS Base URL http://public-repo-1.hortonworks.com/HDP-UTILS-1.1.0.22/repos/centos7
Tarball md5 | asc http://public-repo-1.hortonworks.com/HDP-UTILS-1.1.0.22/repos/centos7/HDP-UTILS-1.1.0.22-centos7.tar.gz
HDP-GPL URL http://public-repo-1.hortonworks.com/HDP-GPL/centos7/3.x/updates/3.1.0.0/hdp.gpl.repo
Tarball md5 | asc http://public-repo-1.hortonworks.com/HDP-GPL/centos7/3.x/updates/3.1.0.0/HDP-GPL-3.1.0.0-centos7-gpl.tar.gz

amazonlinux2

 HDP-3.1.0.0 HDP Version Definition File (VDF) http://public-repo-1.hortonworks.com/HDP/amazonlinux2/3.x/updates/3.1.0.0/HDP-3.1.0.0-1634.xml
Base URL http://public-repo-1.hortonworks.com/HDP/amazonlinux2/3.x/updates/3.1.0.0
Repo File http://public-repo-1.hortonworks.com/HDP/amazonlinux2/3.x/updates/3.1.0.0/hdp.repo
Tarball md5 | asc http://public-repo-1.hortonworks.com/HDP/amazonlinux2/3.x/updates/3.1.0.0/HDP-3.1.0.0-amazonlinux2-rpm.tar.gz
HDP-UTILS Base URL http://public-repo-1.hortonworks.com/HDP-UTILS-1.1.0.22/repos/amazonlinux2
Tarball md5 | asc http://public-repo-1.hortonworks.com/HDP-UTILS-1.1.0.22/repos/amazonlinux2/HDP-UTILS-1.1.0.22-amazonlinux2.tar.gz
HDP-GPL URL http://public-repo-1.hortonworks.com/HDP-GPL/amazonlinux2/3.x/updates/3.1.0.0/hdp.gpl.repo
Tarball md5 | asc http://public-repo-1.hortonworks.com/HDP-GPL/amazonlinux2/3.x/updates/3.1.0.0/HDP-GPL-3.1.0.0-amazonlinux2-gpl.tar.gz
SLES 12 HDP-3.1.0.0 HDP Version Definition File (VDF) http://public-repo-1.hortonworks.com/HDP/sles12/3.x/updates/3.1.0.0/HDP-3.1.0.0-78.xml
Base URL http://public-repo-1.hortonworks.com/HDP/sles12/3.x/updates/3.1.0.0/
Repo File http://public-repo-1.hortonworks.com/HDP/sles12/3.x/updates/3.1.0.0/hdp.repo
Tarball md5 | asc http://public-repo-1.hortonworks.com/HDP/sles12/3.x/updates/3.1.0.0/HDP-3.1.0.0-sles12-rpm.tar.gz
HDP-UTILS Base URL http://public-repo-1.hortonworks.com/HDP-UTILS-1.1.0.22/repos/sles12
Tarball md5 | asc http://public-repo-1.hortonworks.com/HDP-UTILS-1.1.0.22/repos/sles12/HDP-UTILS-1.1.0.22-sles12.tar.gz
HDP-GPL URL http://public-repo-1.hortonworks.com/HDP-GPL/sles12/3.x/updates/3.1.0.0/hdp.gpl.repo
Tarball md5 | asc http://public-repo-1.hortonworks.com/HDP-GPL/sles12/3.x/updates/3.1.0.0/HDP-GPL-3.1.0.0-sles12-gpl.tar.gz
Ubuntu 14 HDP-3.1.0.0 HDP Version Definition File (VDF) http://public-repo-1.hortonworks.com/HDP/ubuntu14/3.x/updates/3.1.0.0/HDP-3.1.0.0-78.xml
Base URL http://public-repo-1.hortonworks.com/HDP/ubuntu14/3.x/updates/3.1.0.0/
Repo File http://public-repo-1.hortonworks.com/HDP/ubuntu14/3.x/updates/3.1.0.0/hdp.list
Tarball md5 | asc http://public-repo-1.hortonworks.com/HDP/ubuntu14/3.x/updates/3.1.0.0/HDP-3.1.0.0-ubuntu14-deb.tar.gz
HDP-UTILS Base URL http://public-repo-1.hortonworks.com/HDP-UTILS-1.1.0.22/repos/ubuntu14
Tarball md5 | asc http://public-repo-1.hortonworks.com/HDP-UTILS-1.1.0.22/repos/ubuntu14/HDP-UTILS-1.1.0.22-ubuntu14.tar.gz
HDP-GPL URL http://public-repo-1.hortonworks.com/HDP-GPL/ubuntu14/3.x/updates/3.1.0.0/hdp.gpl.list
Tarball md5 | asc http://public-repo-1.hortonworks.com/HDP-GPL/ubuntu14/3.x/updates/3.1.0.0/HDP-GPL-3.1.0.0-ubuntu14-gpl.tar.gz
Ubuntu 16 HDP-3.1.0.0 HDP Version Definition File (VDF) http://public-repo-1.hortonworks.com/HDP/ubuntu16/3.x/updates/3.1.0.0/HDP-3.1.0.0-78.xml
Base URL http://public-repo-1.hortonworks.com/HDP/ubuntu16/3.x/updates/3.1.0.0/
Repo File http://public-repo-1.hortonworks.com/HDP/ubuntu16/3.x/updates/3.1.0.0/hdp.list
Tarball md5 | asc http://public-repo-1.hortonworks.com/HDP/ubuntu16/3.x/updates/3.1.0.0/HDP-3.1.0.0-ubuntu16-deb.tar.gz
HDP-UTILS Base URL http://public-repo-1.hortonworks.com/HDP-UTILS-1.1.0.22/repos/ubuntu16
Tarball md5 | asc http://public-repo-1.hortonworks.com/HDP-UTILS-1.1.0.22/repos/ubuntu16/HDP-UTILS-1.1.0.22-ubuntu16.tar.gz
HDP-GPL URL http://public-repo-1.hortonworks.com/HDP-GPL/ubuntu16/3.x/updates/3.1.0.0/hdp.gpl.list
Tarball md5 | asc http://public-repo-1.hortonworks.com/HDP-GPL/ubuntu16/3.x/updates/3.1.0.0/HDP-GPL-3.1.0.0-ubuntu16-gpl.tar.gz
Ubuntu 18 HDP-3.1.0.0 HDP Version Definition File (VDF) http://public-repo-1.hortonworks.com/HDP/ubuntu18/3.x/updates/3.1.0.0/HDP-3.1.0.0-78.xml
Base URL http://public-repo-1.hortonworks.com/HDP/ubuntu18/3.x/updates/3.1.0.0/
Repo File http://public-repo-1.hortonworks.com/HDP/ubuntu18/3.x/updates/3.1.0.0/hdp.list
Tarball md5 | asc http://public-repo-1.hortonworks.com/HDP/ubuntu18/3.x/updates/3.1.0.0/HDP-3.1.0.0-ubuntu18-deb.tar.gz
HDP-UTILS Base URL http://public-repo-1.hortonworks.com/HDP-UTILS-1.1.0.22/repos/ubuntu18
Tarball md5 | asc http://public-repo-1.hortonworks.com/HDP-UTILS-1.1.0.22/repos/ubuntu18/HDP-UTILS-1.1.0.22-ubuntu18.tar.gz
HDP-GPL URL http://public-repo-1.hortonworks.com/HDP-GPL/ubuntu18/3.x/updates/3.1.0.0/hdp.gpl.list
Tarball md5 | asc http://public-repo-1.hortonworks.com/HDP-GPL/ubuntu18/3.x/updates/3.1.0.0/HDP-GPL-3.1.0.0-ubuntu18-gpl.tar.gz
Debian9 HDP-3.1.0.0 HDP Version Definition File (VDF) http://public-repo-1.hortonworks.com/HDP/debian9/3.x/updates/3.1.0.0/HDP-3.1.0.0-78.xml
Base URL http://public-repo-1.hortonworks.com/HDP/debian9/3.x/updates/3.1.0.0/
Repo File http://public-repo-1.hortonworks.com/HDP/debian9/3.x/updates/3.1.0.0/hdp.list
Tarball md5 | asc http://public-repo-1.hortonworks.com/HDP/debian9/3.x/updates/3.1.0.0/HDP-3.1.0.0-debian9-deb.tar.gz
HDP-UTILS Base URL http://public-repo-1.hortonworks.com/HDP-UTILS-1.1.0.22/repos/debian9
Tarball md5 | asc http://public-repo-1.hortonworks.com/HDP-UTILS-1.1.0.22/repos/debian9/HDP-UTILS-1.1.0.22-debian9.tar.gz
HDP-GPL URL http://public-repo-1.hortonworks.com/HDP-GPL/debian9/3.x/updates/3.1.0.0/hdp.gpl.list
Tarball md5 | asc http://public-repo-1.hortonworks.com/HDP-GPL/debian9/3.x/updates/3.1.0.0/HDP-GPL-3.1.0.0-debian9-gpl.tar.gz

'Study > Bigdata' 카테고리의 다른 글

NIFI could not load known_hosts 해결방법  (0) 2020.01.17
NBP Ncloud Cloud Hadoop에서 pySpark로 Object Storage 읽오는 방법  (0) 2019.12.18
python 3.7.5 오픈소스 워크플로우 엔진 apache airflow(celery,rabbitmq,postgresql) 구축하기  (0) 2019.11.26
apache phoenix org.apache.phoenix.exception.PhoenixIOException: org.apache.hadoop.hbase.security.AccessDeniedException: Insufficient permissions for user jdbc  (0) 2018.12.12
Pyspark로 Spark on Yarn Code --1(개발환경구성)  (0) 2018.11.29
HDP3 에서 Spark 로 Hive Table 를 조회했는데 빈값이 나온경우  (0) 2018.10.03

'Study/Bigdata' Related Articles

티스토리툴바