인트라넷(폐쇄망) 환경에서 Ambari, HDP 배포하기

최근, 폐쇄망 환경에서 Ambari,HDP를 배포할 일이 있었습니다. 

조만간 또할것 같지만,  사실 예전에 몇번 해보긴 했는데 오랜만에 하느라 기억이 버벅였네요

 

환경은 다음과 같습니다

centos 7 64bit Server들

 

기본적으로 인터넷환경에서는 Ambari Server 설치후 Ambari Agent가 설치되면서 외부 HDP 레포에서 YUM으로 다운로드 하기 때문에 인터넷이 필요합니다.

 

그럼 폐쇄망 환경은 어떻게 하느냐, 아마 당연하겠지만 , 내부 레포를 만드시면 됩니다

 

필요한 내부레포

1. Centos Base Repo

2. Ambari Repo

3. HDP Repo

 

예전에는 Base레포 없이 했던것 같긴한데, Base레포가 필요한 이유는 Ambari Backend DB를 postgres나 별도의 DB없이 Hive Metastore를 설치할때 Mariadb 또는 MySQL을 필요로 하기 때문입니다.

 

이 외에애도 Ambari가 HDP 레포를 배포하면서 redhat-lsb, python-devel 그리고.. 몇가지 패키지를 기본으로 필요로 하기때문에 Centos Base Repo를 준비해야합니다 

 

Repo를 구축하는 방법은 간단히 구글에서 검색하면 나오기때문에 이부분은 생략하겠습니다

 

일단 HDP3 버전을 기준으로 하자면  (2019년 5월 6일 기준, 최신버전으로 ) 다음과 같습니다

 

 

------------------------------------------------------------------------------------------레포정보

 

Ambari 레포는 다음과 같이 받을수 있고

https://docs.hortonworks.com/HDPDocuments/Ambari-2.7.3.0/bk_ambari-installation/content/ambari_repositories.html

 

 

 

Ambari Repositories

If you do not have Internet access, use the link appropriate for your OS family to download a tarball that contains the software for setting up Ambari.

If you have temporary Internet access, use the link appropriate for your OS family to download a repository file that contains the software for setting up Ambari.

Ambari 2.7.3 Repositories

OS	Format	URL
RedHat 7 CentOS 7 Oracle Linux 7	Base URL	http://public-repo-1.hortonworks.com/ambari/centos7/2.x/updates/2.7.3.0
	Repo File	http://public-repo-1.hortonworks.com/ambari/centos7/2.x/updates/2.7.3.0/ambari.repo
	Tarball md5 |asc	http://public-repo-1.hortonworks.com/ambari/centos7/2.x/updates/2.7.3.0/ambari-2.7.3.0-centos7.tar.gz
amazonlinux 2	Base URL	http://public-repo-1.hortonworks.com/ambari/amazonlinux2/2.x/updates/2.7.3.0
	Repo File	http://public-repo-1.hortonworks.com/ambari/amazonlinux2/2.x/updates/2.7.3.0/ambari.repo
	Tarball md5 |asc	http://public-repo-1.hortonworks.com/ambari/amazonlinux2/2.x/updates/2.7.3.0/ambari-2.7.3.0-amazonlinux2.tar.gz
SLES 12	Base URL	http://public-repo-1.hortonworks.com/ambari/sles12/2.x/updates/2.7.3.0
	Repo File	http://public-repo-1.hortonworks.com/ambari/sles12/2.x/updates/2.7.3.0/ambari.repo
	Tarball md5 |asc	http://public-repo-1.hortonworks.com/ambari/sles12/2.x/updates/2.7.3.0/ambari-2.7.3.0-sles12.tar.gz
Ubuntu 14	Base URL	http://public-repo-1.hortonworks.com/ambari/ubuntu14/2.x/updates/2.7.3.0
	Repo File	http://public-repo-1.hortonworks.com/ambari/ubuntu14/2.x/updates/2.7.3.0/ambari.list
	Tarball md5 |asc	http://public-repo-1.hortonworks.com/ambari/ubuntu14/2.x/updates/2.7.3.0/ambari-2.7.3.0-ubuntu14.tar.gz
Ubuntu 16	Base URL	http://public-repo-1.hortonworks.com/ambari/ubuntu16/2.x/updates/2.7.3.0
	Repo File	http://public-repo-1.hortonworks.com/ambari/ubuntu16/2.x/updates/2.7.3.0/ambari.list
	Tarball md5 |asc	http://public-repo-1.hortonworks.com/ambari/ubuntu16/2.x/updates/2.7.3.0/ambari-2.7.3.0-ubuntu16.tar.gz
Ubuntu 18	Base URL	http://public-repo-1.hortonworks.com/ambari/ubuntu18/2.x/updates/2.7.3.0
	Repo File	http://public-repo-1.hortonworks.com/ambari/ubuntu18/2.x/updates/2.7.3.0/ambari.list
	Tarball md5 |asc	http://public-repo-1.hortonworks.com/ambari/ubuntu18/2.x/updates/2.7.3.0/ambari-2.7.3.0-ubuntu18.tar.gz
Debian 9	Base URL	http://public-repo-1.hortonworks.com/ambari/debian9/2.x/updates/2.7.3.0
	Repo File	http://public-repo-1.hortonworks.com/ambari/debian9/2.x/updates/2.7.3.0/ambari.list
	Tarball md5 |asc	http://public-repo-1.hortonworks.com/ambari/debian9/2.x/updates/2.7.3.0/ambari-2.7.3.0-debian9.tar.gz

HDP 레포는 다음과 같습니다 

https://docs.hortonworks.com/HDPDocuments/Ambari-2.7.3.0/bk_ambari-installation/content/hdp_31_repositories.html

HDP 3.1.0 Repositories - Hortonworks Data Platform

 

 

HDP 3.1.0 Repositories

OS	Version Number	Repository Name	Format	URL
RedHat 7 CentOS 7 Oracle Linux 7	HDP-3.1.0.0	HDP	Version Definition File (VDF)	http://public-repo-1.hortonworks.com/HDP/centos7/3.x/updates/3.1.0.0/HDP-3.1.0.0-78.xml
			Base URL	http://public-repo-1.hortonworks.com/HDP/centos7/3.x/updates/3.1.0.0
			Repo File	http://public-repo-1.hortonworks.com/HDP/centos7/3.x/updates/3.1.0.0/hdp.repo
			Tarball md5 | asc	http://public-repo-1.hortonworks.com/HDP/centos7/3.x/updates/3.1.0.0/HDP-3.1.0.0-centos7-rpm.tar.gz
		HDP-UTILS	Base URL	http://public-repo-1.hortonworks.com/HDP-UTILS-1.1.0.22/repos/centos7
			Tarball md5 | asc	http://public-repo-1.hortonworks.com/HDP-UTILS-1.1.0.22/repos/centos7/HDP-UTILS-1.1.0.22-centos7.tar.gz
		HDP-GPL	URL	http://public-repo-1.hortonworks.com/HDP-GPL/centos7/3.x/updates/3.1.0.0/hdp.gpl.repo
			Tarball md5 | asc	http://public-repo-1.hortonworks.com/HDP-GPL/centos7/3.x/updates/3.1.0.0/HDP-GPL-3.1.0.0-centos7-gpl.tar.gz
amazonlinux2	HDP-3.1.0.0	HDP	Version Definition File (VDF)	http://public-repo-1.hortonworks.com/HDP/amazonlinux2/3.x/updates/3.1.0.0/HDP-3.1.0.0-1634.xml
			Base URL	http://public-repo-1.hortonworks.com/HDP/amazonlinux2/3.x/updates/3.1.0.0
			Repo File	http://public-repo-1.hortonworks.com/HDP/amazonlinux2/3.x/updates/3.1.0.0/hdp.repo
			Tarball md5 | asc	http://public-repo-1.hortonworks.com/HDP/amazonlinux2/3.x/updates/3.1.0.0/HDP-3.1.0.0-amazonlinux2-rpm.tar.gz
		HDP-UTILS	Base URL	http://public-repo-1.hortonworks.com/HDP-UTILS-1.1.0.22/repos/amazonlinux2
			Tarball md5 | asc	http://public-repo-1.hortonworks.com/HDP-UTILS-1.1.0.22/repos/amazonlinux2/HDP-UTILS-1.1.0.22-amazonlinux2.tar.gz
		HDP-GPL	URL	http://public-repo-1.hortonworks.com/HDP-GPL/amazonlinux2/3.x/updates/3.1.0.0/hdp.gpl.repo
			Tarball md5 | asc	http://public-repo-1.hortonworks.com/HDP-GPL/amazonlinux2/3.x/updates/3.1.0.0/HDP-GPL-3.1.0.0-amazonlinux2-gpl.tar.gz
SLES 12	HDP-3.1.0.0	HDP	Version Definition File (VDF)	http://public-repo-1.hortonworks.com/HDP/sles12/3.x/updates/3.1.0.0/HDP-3.1.0.0-78.xml
			Base URL	http://public-repo-1.hortonworks.com/HDP/sles12/3.x/updates/3.1.0.0/
			Repo File	http://public-repo-1.hortonworks.com/HDP/sles12/3.x/updates/3.1.0.0/hdp.repo
			Tarball md5 | asc	http://public-repo-1.hortonworks.com/HDP/sles12/3.x/updates/3.1.0.0/HDP-3.1.0.0-sles12-rpm.tar.gz
		HDP-UTILS	Base URL	http://public-repo-1.hortonworks.com/HDP-UTILS-1.1.0.22/repos/sles12
			Tarball md5 | asc	http://public-repo-1.hortonworks.com/HDP-UTILS-1.1.0.22/repos/sles12/HDP-UTILS-1.1.0.22-sles12.tar.gz
		HDP-GPL	URL	http://public-repo-1.hortonworks.com/HDP-GPL/sles12/3.x/updates/3.1.0.0/hdp.gpl.repo
			Tarball md5 | asc	http://public-repo-1.hortonworks.com/HDP-GPL/sles12/3.x/updates/3.1.0.0/HDP-GPL-3.1.0.0-sles12-gpl.tar.gz
Ubuntu 14	HDP-3.1.0.0	HDP	Version Definition File (VDF)	http://public-repo-1.hortonworks.com/HDP/ubuntu14/3.x/updates/3.1.0.0/HDP-3.1.0.0-78.xml
			Base URL	http://public-repo-1.hortonworks.com/HDP/ubuntu14/3.x/updates/3.1.0.0/
			Repo File	http://public-repo-1.hortonworks.com/HDP/ubuntu14/3.x/updates/3.1.0.0/hdp.list
			Tarball md5 | asc	http://public-repo-1.hortonworks.com/HDP/ubuntu14/3.x/updates/3.1.0.0/HDP-3.1.0.0-ubuntu14-deb.tar.gz
		HDP-UTILS	Base URL	http://public-repo-1.hortonworks.com/HDP-UTILS-1.1.0.22/repos/ubuntu14
			Tarball md5 | asc	http://public-repo-1.hortonworks.com/HDP-UTILS-1.1.0.22/repos/ubuntu14/HDP-UTILS-1.1.0.22-ubuntu14.tar.gz
		HDP-GPL	URL	http://public-repo-1.hortonworks.com/HDP-GPL/ubuntu14/3.x/updates/3.1.0.0/hdp.gpl.list
			Tarball md5 | asc	http://public-repo-1.hortonworks.com/HDP-GPL/ubuntu14/3.x/updates/3.1.0.0/HDP-GPL-3.1.0.0-ubuntu14-gpl.tar.gz
Ubuntu 16	HDP-3.1.0.0	HDP	Version Definition File (VDF)	http://public-repo-1.hortonworks.com/HDP/ubuntu16/3.x/updates/3.1.0.0/HDP-3.1.0.0-78.xml
			Base URL	http://public-repo-1.hortonworks.com/HDP/ubuntu16/3.x/updates/3.1.0.0/
			Repo File	http://public-repo-1.hortonworks.com/HDP/ubuntu16/3.x/updates/3.1.0.0/hdp.list
			Tarball md5 | asc	http://public-repo-1.hortonworks.com/HDP/ubuntu16/3.x/updates/3.1.0.0/HDP-3.1.0.0-ubuntu16-deb.tar.gz
		HDP-UTILS	Base URL	http://public-repo-1.hortonworks.com/HDP-UTILS-1.1.0.22/repos/ubuntu16
			Tarball md5 | asc	http://public-repo-1.hortonworks.com/HDP-UTILS-1.1.0.22/repos/ubuntu16/HDP-UTILS-1.1.0.22-ubuntu16.tar.gz
		HDP-GPL	URL	http://public-repo-1.hortonworks.com/HDP-GPL/ubuntu16/3.x/updates/3.1.0.0/hdp.gpl.list
			Tarball md5 | asc	http://public-repo-1.hortonworks.com/HDP-GPL/ubuntu16/3.x/updates/3.1.0.0/HDP-GPL-3.1.0.0-ubuntu16-gpl.tar.gz
Ubuntu 18	HDP-3.1.0.0	HDP	Version Definition File (VDF)	http://public-repo-1.hortonworks.com/HDP/ubuntu18/3.x/updates/3.1.0.0/HDP-3.1.0.0-78.xml
			Base URL	http://public-repo-1.hortonworks.com/HDP/ubuntu18/3.x/updates/3.1.0.0/
			Repo File	http://public-repo-1.hortonworks.com/HDP/ubuntu18/3.x/updates/3.1.0.0/hdp.list
			Tarball md5 | asc	http://public-repo-1.hortonworks.com/HDP/ubuntu18/3.x/updates/3.1.0.0/HDP-3.1.0.0-ubuntu18-deb.tar.gz
		HDP-UTILS	Base URL	http://public-repo-1.hortonworks.com/HDP-UTILS-1.1.0.22/repos/ubuntu18
			Tarball md5 | asc	http://public-repo-1.hortonworks.com/HDP-UTILS-1.1.0.22/repos/ubuntu18/HDP-UTILS-1.1.0.22-ubuntu18.tar.gz
		HDP-GPL	URL	http://public-repo-1.hortonworks.com/HDP-GPL/ubuntu18/3.x/updates/3.1.0.0/hdp.gpl.list
			Tarball md5 | asc	http://public-repo-1.hortonworks.com/HDP-GPL/ubuntu18/3.x/updates/3.1.0.0/HDP-GPL-3.1.0.0-ubuntu18-gpl.tar.gz
Debian9	HDP-3.1.0.0	HDP	Version Definition File (VDF)	http://public-repo-1.hortonworks.com/HDP/debian9/3.x/updates/3.1.0.0/HDP-3.1.0.0-78.xml
			Base URL	http://public-repo-1.hortonworks.com/HDP/debian9/3.x/updates/3.1.0.0/
			Repo File	http://public-repo-1.hortonworks.com/HDP/debian9/3.x/updates/3.1.0.0/hdp.list
			Tarball md5 | asc	http://public-repo-1.hortonworks.com/HDP/debian9/3.x/updates/3.1.0.0/HDP-3.1.0.0-debian9-deb.tar.gz
		HDP-UTILS	Base URL	http://public-repo-1.hortonworks.com/HDP-UTILS-1.1.0.22/repos/debian9
			Tarball md5 | asc	http://public-repo-1.hortonworks.com/HDP-UTILS-1.1.0.22/repos/debian9/HDP-UTILS-1.1.0.22-debian9.tar.gz
		HDP-GPL	URL	http://public-repo-1.hortonworks.com/HDP-GPL/debian9/3.x/updates/3.1.0.0/hdp.gpl.list
			Tarball md5 | asc	http://public-repo-1.hortonworks.com/HDP-GPL/debian9/3.x/updates/3.1.0.0/HDP-GPL-3.1.0.0-debian9-gpl.tar.gz