编译Hudi1.0.01、Maven软件下载https://maven.apache.org/download.cgi地址https://dlcdn.apache.org/maven/maven-3/3.9.9/binaries/apache-maven-3.9.9-bin.tar.gzwget https://dlcdn.apache.org/maven/maven-3/3.9.9/binaries/apache-maven-3.9.9-bin.tar.gztar -zxvf apache-maven-3.9.9-bin.tar.gz2、添加mvn环境变量编辑环境变量vi /etc/profile环境变量添加export MAVEN_HOME/usr/local/soft/apache-maven-3.9.9 export PATH$PATH:$MAVEN_HOME/bin使环境变量生效source /etc/profile3、添加Maven镜像/usr/local/soft/apache-maven-3.9.9/conf/settings.xml两个都需要只有阿里云有些库下载不了mirror idalimaven/id namealiyun maven/name urlhttp://maven.aliyun.com/nexus/content/groups/public//url mirrorOfcentral/mirrorOf /mirror mirror idconfluent/id nameconfluent maven/name urlhttp://packages.confluent.io/maven//url mirrorOfconfluent/mirrorOf /mirror4、验证mvnmvn -v5、下载hudi 1.0.0hudi下载地址Download | Apache Hudi或Index of /hudi/1.0.0下载wget https://downloads.apache.org/hudi/1.0.0/hudi-1.0.0.src.tgz6、解压huditar -zxvf hudi-1.0.0.src.tgz7、修改hudi源码a、 修改/usr/local/soft/hudi-1.0.0/hudi-sync/hudi-hive-sync/src/test/java/org/apache/hudi/hive/testutils/HiveTestUtil.java文件第250行把 zkServer.shutdown(true);改为 zkServer.shutdown();b、修改/usr/local/soft/hudi-1.0.0/pom.xml注释或去掉410行内容cd /usr/local/soft wget http://packages.confluent.io/archive/5.5/confluent-5.5.0-2.12.zip unzip confluent-5.5.0-2.12.zip cd confluent-5.5.0/ mvn install:install-file -DgroupIdio.confluent -DartifactIdcommon-config -Dversion5.5.0 -Dpackagingjar -Dfile./confluent-5.5.0/share/java/confluent-common/common-config-5.5.0.jar mvn install:install-file -DgroupIdio.confluent -DartifactIdommon-utils -Dversion5.5.0 -Dpackagingjar -Dfile./confluent-5.5.0/share/java/confluent-common/ommon-utils-5.5.0.jar mvn install:install-file -DgroupIdio.confluent -DartifactIdcommon-utils -Dversion5.5.0 -Dpackagingjar -Dfile./confluent-5.5.0/share/java/confluent-common/common-utils-5.5.0.jar mvn install:install-file -DgroupIdio.confluent -DartifactIdkafka-avro-serializer -Dversion5.5.0 -Dpackagingjar -Dfile./confluent-5.5.0/share/java/kafka-rest/kafka-avro-serializer-5.5.0.jar mvn install:install-file -DgroupIdio.confluent -DartifactIdkafka-schema-registry-client -Dversion5.5.0 -Dpackagingjar -Dfile./confluent-5.5.0/share/java/kafka-rest/kafka-schema-registry-client-5.5.0.jar mvn install:install-file -DgroupIdio.confluent -DartifactIdkafka-json-schema-serializer -Dversion5.5.0 -Dpackagingjar -Dfile./confluent-5.5.0/share/java/kafka-rest/kafka-json-schema-serializer-5.5.0.jarc、修改pom添加如下内容/usr/local/soft/hudi-1.0.0/packaging/hudi-spark-bundle/pom.xml/usr/local/soft/hudi-1.0.0/packaging/hudi-utilities-bundle/pom.xml!-- 增加hudi配置版本的jetty -- dependency groupIdorg.eclipse.jetty/groupId artifactIdjetty-server/artifactId version${jetty.version}/version /dependency dependency groupIdorg.eclipse.jetty/groupId artifactIdjetty-util/artifactId version${jetty.version}/version /dependency dependency groupIdorg.eclipse.jetty/groupId artifactIdjetty-webapp/artifactId version${jetty.version}/version /dependency dependency groupIdorg.eclipse.jetty/groupId artifactIdjetty-http/artifactId version${jetty.version}/version /dependency8、编译hudicd hudi-1.0.0mvn clean package -DskipTests -Dspark3.5 -Dflink1.20 -Dscala-2.12 -Dhadoop.version3.4.0 -Pflink-bundle-shade-hive3或mvn clean package -DskipTests -Dspark3.4 -Dflink1.14 -Dscala-2.12 -Dhadoop.version3.1.1 -Pflink-bundle-shade-hive3参考CDP集成Hudi-编译部署-CSDN博客大数据之数据湖Apache Hudi-CSDN博客
