尚硅谷大数据技术之Hadoop(MapReduce)(新)第2章 Hadoop序列化
3.3.7 Combiner合并
(6)自定义Combiner实现步骤
(a)自定义一个Combiner继承Reducer,重写Reduce方法
public class WordcountCombiner extends Reducer<Text, IntWritable, Text,IntWritable>{
@Override protected void reduce(Text key, Iterable<IntWritable> values,Context context) throws IOException, InterruptedException {
// 1 汇总操作 int count = 0; for(IntWritable v :values){ count += v.get(); }
// 2 写出 context.write(key, new IntWritable(count)); } } |
(b)在Job驱动类中设置:
job.setCombinerClass(WordcountCombiner.class); |