Spark大數據商業實戰三部曲 內核解密 商業案例 性能調優pdf

圖書網
圖書網
圖書網
10700
文章
1511
評論
2020年8月5日19:29:00 評論 61

Spark大數據商業實戰三部曲 內核解密 商業案例 性能調優 作者:王家林,段智華,夏陽

Spark大數據商業實戰三部曲 內核解密 商業案例 性能調優 出版社:清華大學出版社

Spark大數據商業實戰三部曲 內核解密 商業案例 性能調優 內容簡介

旧版尊龙人生就是博下载《SparkDASHUJUSHANGYESHIZHANSANBUQU:NEIHEJIEMI|SHANGYEANLI|XINGNENGDIAOYOU》JIYUSpark 2.2.X,YISparkSHANGYEANLISHIZHANHESparkZAISHENGCHANHUANJINGXIAJIHUSUOYOULEIXINGDEXINGNENGDIAOYOUWEIHEXIN,YISparkNEIHEJIEMIWEIJISHI,FENWEISHANGPIAN、ZHONGPIAN、XIAPIAN,DUIQIYESHENGCHANHUANJINGXIADESparkSHANGYEANLIYUXINGNENGDIAOYOUCHOUSIBOJIANDIJINXINGPOUXI。SHANGPIANJIYUSparkYUANMA,CONGYIGEDONGSHOUSHIZHANANLIRUSHOU,XUNXUJIANJINDIQUANMIANJIEXILESpark 2.2XINTEXINGJISparkNEIHEYUANMA;ZHONGPIANXUANQUSparkKAIFAZHONG*JUYOUDAIBIAODEJINGDIANXUEXIANLI,SHENRUQIANCHUDIJIESHAO,ZAIANLIZHONGZONGHEYINGYONGSparkDEDASHUJUJISHU;XIAPIANXINGNENGDIAOYOUNEIRONGJIBENWANQUANFUGAILESparkZAISHENGCHANHUANJINGXIADESUOYOUDIAOYOUJISHU。

Spark大數據商業實戰三部曲 內核解密 商業案例 性能調優 目錄

QIANYAN

SHANGPIAN NEIHEJIEMI

DI1ZHANG DIANGUANGSHIHUOJIANTIYANSpark 2.2KAIFASHIZHAN

1.1 TONGGUORDDSHIZHANDIANYINGDIANPINGXITONGRUMENJIYUANMAYUEDU

1.2 TONGGUODataFrameHEDataSetSHIZHANDIANYINGDIANPINGXITONG

1.3 Spark 2.2YUANMAYUEDUHUANJINGDAJIANJIYUANMAYUEDUTIYAN

DI2ZHANG Spark 2.2JISHUJIYUANLI

2.1 Spark 2.2ZONGSHU

旧版尊龙人生就是博下载2.2 Spark 2.2 Core

旧版尊龙人生就是博下载2.3 Spark 2.2 SQL

旧版尊龙人生就是博下载2.4 Spark 2.2 Streaming

2.5 Spark 2.2 MLlib

旧版尊龙人生就是博下载2.6 Spark 2.2 GraphX

DI3ZHANG SparkDELINGHUN:RDDHEDataSet

旧版尊龙人生就是博下载3.1 WEISHENMESHUORDDHEDataSetSHISparkDELINGHUN

3.2 RDDDANXINGTEXINGQIGEFANGMIANJIEXI

3.3 RDDYILAIGUANXI

3.4 JIEXISparkZHONGDEDAGLUOJISHITU

3.5 RDDNEIBUDEJISUANJIZHI

3.6 Spark RDDRONGCUOYUANLIJIQISIDAHEXINYAODIANJIEXI

3.7 Spark RDDZHONGRuntimeLIUCHENGJIEXI

3.8 TONGGUOWordCountSHIZHANJIEXISpark RDDNEIBUJIZHI

3.9 JIYUDataSetDEDAIMADAODISHIRUHEYIBUBUZHUANHUACHENGWEIRDDDE

DI4ZHANG Spark DriverQIDONGNEIMUPOUXI

4.1 Spark Driver ProgramPOUXI

4.2 DAGSchedulerJIEXI

4.3 TaskSchedulerJIEXI

4.4 SchedulerBackendJIEXI

旧版尊龙人生就是博下载4.5 DATONGSparkXITONGYUNXINGNEIMUJIZHIXUNHUANLIUCHENG

4.6 BENZHANGZONGJIE

DI5ZHANG SparkJIQUNQIDONGYUANLIHEYUANMAXIANGJIE

5.1 MasterQIDONGYUANLIHEYUANMAXIANGJIE

旧版尊龙人生就是博下载5.2 WorkerQIDONGYUANLIHEYUANMAXIANGJIE

旧版尊龙人生就是博下载5.3 ExecutorBackendQIDONGYUANLIHEYUANMAXIANGJIE

5.4 ExecutorZHONGRENWUDEZHIXING

5.5 ExecutorZHIXINGJIEGUODECHULIFANGSHI

5.6 BENZHANGZONGJIE

旧版尊龙人生就是博下载DI6ZHANG Spark ApplicationTIJIAOGEIJIQUNDEYUANLIHEYUANMAXIANGJIE

旧版尊龙人生就是博下载6.1 Spark ApplicationDAODISHIRUHETIJIAOGEIJIQUNDE

6.2 Spark ApplicationSHIRUHEXIANGJIQUNSHENQINGZIYUANDE

6.3 CONGApplicationTIJIAODEJIAODUZHONGXINSHENSHIDriver

6.4 CONGApplicationTIJIAODEJIAODUZHONGXINSHENSHIExecutor

6.5 Spark 1.6 RPCNEIMUJIEMI:YUNXINGJIZHI、YUANMAXIANGJIE、NettyYUAkkaDENG

6.6 BENZHANGZONGJIE

旧版尊龙人生就是博下载DI7ZHANG ShuffleYUANLIHEYUANMAXIANGJIE

7.1 GAISHU

7.2 ShuffleDEKUANGJIA

7.3 Hash Based Shuffle

7.4 Sorted Based Shuffle

7.5 Tungsten Sorted Based Shuffle

旧版尊龙人生就是博下载7.6 ShuffleYUStorageMOKUAIJIANDEJIAOHU

7.7 BENZHANGZONGJIE

旧版尊龙人生就是博下载DI8ZHANG JobGONGZUOYUANLIHEYUANMAXIANGJIE

8.1 JobDAODIZAISHENMESHIHOUCHANSHENG

8.2 StageHUAFENNEIMU

8.3 TaskQUANSHENGMINGZHOUQIXIANGJIE

8.4 ShuffleMapTaskHEResultTaskCHULIJIEGUOSHIRUHEBEIDriverGUANLIDE

DI9ZHANG SparkZHONGCacheHEcheckpointYUANLIHEYUANMAXIANGJIE

旧版尊龙人生就是博下载9.1 SparkZHONGCacheYUANLIHEYUANMAXIANGJIE

9.2 SparkZHONGcheckpointYUANLIHEYUANMAXIANGJIE

DI10ZHANG SparkZHONGBroadcastHEAccumulatorYUANLIHEYUANMAXIANGJIE

10.1 SparkZHONGBroadcastYUANLIHEYUANMAXIANGJIE

10.2 SparkZHONGAccumulatorYUANLIHEYUANMAXIANGJIE

DI11ZHANG SparkYUDASHUJUQITAJINGDIANZUJIANZHENGHEYUANLIYUSHIZHAN

11.1 SparkZUJIANZONGHEYINGYONG

11.2 SparkYUAlluxioZHENGHEYUANLIYUSHIZHAN

旧版尊龙人生就是博下载11.3 SparkYUJob ServerZHENGHEYUANLIYUSHIZHAN

11.4 SparkYURedisZHENGHEYUANLIYUSHIZHAN

ZHONGPIAN SHANGYEANLI

DI12ZHANG SparkSHANGYEANLIZHIDASHUJUDIANYINGDIANPINGXITONGYINGYONGANLI

12.1 TONGGUORDDSHIXIANFENXIDIANYINGDEYONGHUXINGWEIXINXI

旧版尊龙人生就是博下载12.2 TONGGUORDDSHIXIANDIANYINGLIUXINGDUFENXI

12.3 TONGGUORDDFENXIGEZHONGLEIXINGDEZUIXIAIDIANYINGTopNJIXINGNENGYOUHUAJIQIAO

旧版尊龙人生就是博下载12.4 TONGGUORDDFENXIDIANYINGDIANPINGXITONGFANGQQHEWEIXINDENGYONGHUQUNFENXIJIGUANGBOBEIHOUJIZHIJIEMI

旧版尊龙人生就是博下载12.5 TONGGUORDDFENXIDIANYINGDIANPINGXITONGSHIXIANJavaHEScalaBANBENDEERCIPAIXUXITONG

旧版尊龙人生就是博下载12.6 TONGGUOSpark SQLZHONGDESQLYUJUSHIXIANDIANYINGDIANPINGXITONGYONGHUXINGWEIFENXI

12.7 TONGGUOSpark SQLXIADELIANGZHONGBUTONGFANGSHISHIXIANKOUBEIZUIJIADIANYINGFENXI

12.8 TONGGUOSpark SQLXIADELIANGZHONGBUTONGFANGSHISHIXIANZUILIUXINGDIANYINGFENXI

旧版尊龙人生就是博下载12.9 TONGGUODataFrameFENXIZUISHOUNANXINGHENVXINGXIAIDIANYINGTopN

旧版尊龙人生就是博下载12.10 CHUNCUITONGGUODataFrameFENXIDIANYINGDIANPINGXITONGFANGQQHEWEIXIN、TAOBAODENGYONGHUQUN

旧版尊龙人生就是博下载12.11 CHUNCUITONGGUODataSetDUIDIANYINGDIANPINGXITONGJINXINGLIUXINGDUHEBUTONGNIANLINGJIEDUANXINGQUFENXIDENG

12.12 DASHUJUDIANYINGDIANPINGXITONGYINGYONGANLISHEJIDEHEXINZHISHIDIANYUANLI、YUANMAJIANLIDAIMA

12.13 BENZHANGZONGJIE

旧版尊龙人生就是博下载DI13ZHANG Spark 2.2SHIZHANZHIDatasetKAIFASHIZHANQIYERENYUANGUANLIXITONGYINGYONGANLI

13.1 QIYERENYUANGUANLIXITONGYINGYONGANLIYEWUXUQIUFENXI

13.2 QIYERENYUANGUANLIXITONGYINGYONGANLISHUJUJIANMO

旧版尊龙人生就是博下载13.3 TONGGUOSparkSessionCHUANGJIANANLIKAIFASHIZHANSHANGXIAWENHUANJING

旧版尊龙人生就是博下载13.4 TONGGUOmap、flatMap、mapPartitionsDENGFENXIQIYERENYUANGUANLIXITONG

旧版尊龙人生就是博下载13.5 TONGGUOdropDuplicate、coalesce、repartitionDENGFENXIQIYERENYUANGUANLIXITONG

13.6 TONGGUOsort、join、joinWithDENGFENXIQIYERENYUANGUANLIXITONG

13.7 TONGGUOrandomSplit、sample、selectDENGFENXIQIYERENYUANGUANLIXITONG

13.8 TONGGUOgroupBy、agg、colDENGFENXIQIYERENYUANGUANLIXITONG

13.9 TONGGUOcollect_list、collect_setDENGFENXIQIYERENYUANGUANLIXITONG

旧版尊龙人生就是博下载13.10 TONGGUOavg、sum、countDistinctDENGFENXIQIYERENYUANGUANLIXITONG

13.11 DatasetKAIFASHIZHANQIYERENYUANGUANLIXITONGYINGYONGANLIDAIMA

13.12 BENZHANGZONGJIE

DI14ZHANG SparkSHANGYEANLIZHIDIANSHANGJIAOHUSHIFENXIXITONGYINGYONGANLI

旧版尊龙人生就是博下载14.1 CHUNCUITONGGUODataSetJINXINGDIANSHANGJIAOHUSHIFENXIXITONGZHONGTEDINGSHIDUANFANGWENCISHUTopN

14.2 CHUNCUITONGGUODataSetFENXITEDINGSHIDUANGOUMAIJINETop10HEFANGWENCISHUZENGCHANGTop10

旧版尊龙人生就是博下载14.3 CHUNCUITONGGUODataSetJINXINGDIANSHANGJIAOHUSHIFENXIXITONGZHONGGEZHONGLEIXINGTopNFENXISHIZHANXIANGJIE

旧版尊龙人生就是博下载14.4 DIANSHANGJIAOHUSHIFENXIXITONGYINGYONGANLISHEJIDEHEXINZHISHIDIANYUANLI、YUANMAJIANLIDAIMA

14.5 BENZHANGZONGJIE

DI15ZHANG SparkSHANGYEANLIZHINBALANQIUYUNDONGYUANDASHUJUFENXIXITONGYINGYONGANLI

旧版尊龙人生就是博下载15.1 NBALANQIUYUNDONGYUANDASHUJUFENXIXITONGJIAGOUHESHIXIANSILU

15.2 NBALANQIUYUNDONGYUANDASHUJUFENXIXITONGDAIMASHIZHAN:SHUJUQINGXIHECHUBUCHULI

15.3 NBALANQIUYUNDONGYUANDASHUJUFENXIDAIMASHIZHANZHIHEXINJICHUSHUJUXIANGBIANXIE

15.4 NBALANQIUYUNDONGYUANDASHUJUFENXIWANZHENGDAIMACESHIHESHIZHAN

旧版尊龙人生就是博下载15.5 NBALANQIUYUNDONGYUANDASHUJUFENXIXITONGYINGYONGANLISHEJIDEHEXINZHISHIDIAN、YUANLI、YUANMA

15.6 BENZHANGZONGJIE

DI16ZHANG DIANSHANGGUANGGAODIANJIDASHUJUSHISHILIUCHULIXITONGANLI

16.1 DIANSHANGGUANGGAODIANJIZONGHEANLIXUQIUFENXIHEJISHUJIAGOU

16.2 DIANSHANGGUANGGAODIANJIZONGHEANLIZAIXIANDIANJITONGJISHIZHAN

旧版尊龙人生就是博下载16.3 DIANSHANGGUANGGAODIANJIZONGHEANLIHEIMINGDANGUOLVSHIXIAN

16.4 DIANSHANGGUANGGAODIANJIZONGHEANLIDICENGSHUJUCENGDEJIANMOHEBIANMASHIXIAN(JIYUMySQL)

旧版尊龙人生就是博下载16.5 DIANSHANGGUANGGAODIANJIZONGHEANLIDONGTAIHEIMINGDANGUOLVZHENZHENGDESHIXIANDAIMA

旧版尊龙人生就是博下载16.6 DONGTAIHEIMINGDANJIYUSHUJUKUMySQLDEZHENZHENGCAOZUODAIMASHIZHAN

旧版尊龙人生就是博下载16.7 TONGGUOupdateStateByKeyDENGSHIXIANGUANGGAODIANJILIULIANGDEZAIXIANGENGXINTONGJI

16.8 SHIXIANMEIGESHENGFENDIANJIPAIMINGTop5GUANGGAO

旧版尊龙人生就是博下载16.9 SHIXIANGUANGGAODIANJITrendQUSHIJISUANSHIZHAN

16.10 SHIZHANMONIDIANJISHUJUDESHENGCHENGHESHUJUBIAOSQLDEJIANLI

旧版尊龙人生就是博下载16.11 DIANSHANGGUANGGAODIANJIZONGHEANLIYUNXINGJIEGUO

16.12 DIANSHANGGUANGGAODIANJIZONGHEANLIScalaBANBENGUANZHUDIAN

16.13 DIANSHANGGUANGGAODIANJIZONGHEANLIKECHENGDEJavaYUANMA

16.14 DIANSHANGGUANGGAODIANJIZONGHEANLIKECHENGDEScalaYUANMA

16.15 BENZHANGZONGJIE

DI17ZHANG SparkZAITONGXINYUNYINGSHANGSHENGCHANHUANJINGZHONGDEYINGYONGANLI

旧版尊龙人生就是博下载17.1 SparkZAITONGXINYUNYINGSHANGRONGHEZHIFUXITONGRIZHITONGJIFENXIZHONGDEZONGHEYINGYONGANLI

旧版尊龙人生就是博下载17.2 SparkZAIGUANGKUANYONGHULIULIANGRELIFENBUGISXITONGZHONGDEZONGHEYINGYONGANLI

17.3 BENZHANGZONGJIE

旧版尊龙人生就是博下载DI18ZHANG SHIYONGSpark GraphXSHIXIANHUNLIANSHEJIAOWANGLUODUOWEIDUFENXIANLI

18.1 Spark GraphXFAZHANYANBIANLISHIHEZAIYEJIEDESHIYONGANLI

18.2 Spark GraphXSHEJISHIXIANDEHEXINYUANLI

18.3 Table operatorHEGraph Operator

18.4 Vertices、edges、triplets

18.5 YIZUIYUANSHIDEFANGSHIGOUJIANGraph

旧版尊龙人生就是博下载18.6 DIYIGEGraphDAIMASHILIBINGJINXINGVertices、edges、tripletsCAOZUOSHIZHAN

18.7 SHUJUJIAZAICHENGWEIGraphBINGJINXINGCAOZUOSHIZHAN

旧版尊龙人生就是博下载18.8 TUCAOZUOZHIProperty OperatorsSHIZHAN

18.9 TUCAOZUOZHIStructural OperatorsSHIZHAN

18.10 TUCAOZUOZHIComputing DegreeSHIZHAN

18.11 TUCAOZUOZHICollecting NeighborsSHIZHAN

18.12 TUCAOZUOZHIJoin OperatorsSHIZHAN

18.13 TUCAOZUOZHIaggregateMessagesSHIZHAN

旧版尊龙人生就是博下载18.14 TUSUANFAZHIPregel APIYUANLIJIEXIYUSHIZHAN

18.15 TUSUANFAZHIShortestPathsYUANLIJIEXIYUSHIZHAN

旧版尊龙人生就是博下载18.16 TUSUANFAZHIPageRankYUANLIJIEXIYUSHIZHAN

旧版尊龙人生就是博下载18.17 TUSUANFAZHITriangleCountYUANLIJIEXIYUSHIZHAN

18.18 SHIYONGSpark GraphXSHIXIANHUNLIANSHEJIAOWANGLUODUOWEIDUFENXISHIZHAN

旧版尊龙人生就是博下载18.19 HUNLIANSHEJIAOWANGLUODUOWEIDUFENXIANLIDAIMA

18.20 BENZHANGZONGJIE

XIAPIAN XINGNENGDIAOYOU

DI19ZHANG DUIYUNXINGZAIYARNSHANGDESparkJINXINGXINGNENGDIAOYOU

19.1 YUNXINGHUANJINGJarBAOGUANLIJISHUJUBENDIXINGYUANLIDIAOYOUSHIJIAN

19.2 Spark on YARNLIANGZHONGBUTONGDEDIAODUMOXINGJIQIDIAOYOU

19.3 YARNDUILIEZIYUANBUZUYINQIDESparkYINGYONGCHENGXUSHIBAIDEYUANYINJIDIAOYOUFANGAN

19.4 Spark on YARNMOSHIXIAExecutorJINGCHANGBEISHASIDEYUANYINJIDIAOYOUFANGAN

19.5 YARN-ClientMOSHIXIAWANGKALIULIANGJIZENGDEYUANYINJIDIAOYOUFANGAN

旧版尊龙人生就是博下载19.6 YARN-ClusterMOSHIXIAJVMZHANNEICUNYICHUDEYUANYINJIDIAOYOUFANGAN

DI20ZHANG SparkSUANZIDIAOYOUZUIJIASHIJIAN

20.1 SHIYONGmapPartitionsHUOZHEmapPartitionWithIndexQUDAImapCAOZUO

20.2 SHIYONGforeachPartitionBASparkSHUJUCHIJIUHUADAOWAIBUCUNCHUJIEZHI

旧版尊龙人生就是博下载20.3 SHIYONGcoalesceQUDAIrePartitionCAOZUO

20.4 SHIYONGrepartitionAndSortWithinPartitionsQUDAIrepartitionHEsortDELIANHECAOZUO

旧版尊龙人生就是博下载20.5 SHIYONGtreeReduceQUDAIreduceDEYUANLIHEYUANMA

旧版尊龙人生就是博下载20.6 SHIYONGtreeAggregateQUDAIAggregateDEYUANLIHEYUANMA

旧版尊龙人生就是博下载20.7 reduceByKeyGAOXIAOYUNXINGDEYUANLIHEYUANMAJIEMI

旧版尊龙人生就是博下载20.8 SHIYONGAggregateByKeyQUDAIgroupByKeyDEYUANLIHEYUANMA

旧版尊龙人生就是博下载20.9 JoinBUCHANSHENGShuffleDEQINGKUANGJIANLISHIZHAN

20.10 RDDFUYONGXINGNENGDIAOYOUZUIJIASHIJIAN

旧版尊龙人生就是博下载DI21ZHANG SparkPINFANYUDAODEXINGNENGWENTIJIDIAOYOUJIQIAO

旧版尊龙人生就是博下载21.1 SHIYONGBroadCastGUANGBODABIANLIANGHEYEWUPEIZHIXINXIYUANLIHEANLISHIZHAN

21.2 SHIYONGKryoQUDAIScalaMORENDEXULIEQIYUANLIHEANLISHIZHAN

21.3 SHIYONGFastUtilYOUHUAJVMSHUJUGESHIJIEXIHEANLISHIZHAN

21.4 PersistJIcheckpointSHIYONGSHIDEZHENGWUFANGSHI

21.5 XULIEHUADAOZHIDEBAOCUOYUANYINJIEXIHEDIAOYOUSHIZHAN

21.6 SUANZIFANHUINULLCHANSHENGDEWENTIJIJIEJUEBANFA

DI22ZHANG SparkJIQUNZIYUANFENPEIJIBINGXINGDUDIAOYOUZUIJIASHIJIAN

22.1 SHIJISHENGCHANHUANJINGXIAMEIGEExecutorNEICUNJICPUDEJUTIPEIZHIJIYUANYIN

22.2 SparkBINGXINGDUSHEZHIZUIJIASHIJIAN

旧版尊龙人生就是博下载DI23ZHANG SparkJIQUNZHONGMapperDUAN、ReducerDUANNEICUNDIAOYOU

23.1 SparkJIQUNZHONGMapperDUANNEICUNDIAOYOUSHIZHAN

23.2 SparkJIQUNZHONGReducerDUANNEICUNDIAOYOUSHIZHAN

旧版尊龙人生就是博下载DI24ZHANG SHIYONGBroadcastSHIXIANMapperDUANShuffleJUHEGONGNENGDEYUANLIHEDIAOYOUSHIZHAN

旧版尊龙人生就是博下载24.1 SHIYONGBroadcastSHIXIANMapperDUANShuffleJUHEGONGNENGDEYUANLI

旧版尊龙人生就是博下载24.2 SHIYONGBroadcastSHIXIANMapperDUANShuffleJUHEGONGNENGDIAOYOUSHIZHAN

DI25ZHANG SHIYONGAccumulatorGAOXIAODISHIXIANFENBUSHIJIQUNQUANJUJISHUQIDEYUANLIHEDIAOYOUANLI

25.1 AccumulatorNEIBUGONGZUOYUANLI

旧版尊龙人生就是博下载25.2 AccumulatorZIDINGYISHIXIANYUANLIHEYUANMAJIEXI

旧版尊龙人生就是博下载25.3 AccumulatorZUOQUANJUJISHUQIANLISHIZHAN

旧版尊龙人生就是博下载DI26ZHANG SparkXIAJVMXINGNENGDIAOYOUZUIJIASHIJIAN

26.1 JVMNEICUNJIAGOUXIANGJIEJIDIAOYOU

旧版尊龙人生就是博下载26.2 SparkZHONGDUIJVMSHIYONGDENEICUNYUANLITUXIANGJIEJIDIAOYOU

旧版尊龙人生就是博下载26.3 SparkXIAJVMDEOn-HeapHEOff-HeapJIEMI

26.4 SparkXIADEJVM GCDAOZHIDEShuffleLAQUWENJIANSHIBAIJIDIAOYOUFANGAN

26.5 SparkXIADEExecutorDUIJVMDUIWAINEICUNLIANJIEDENGDAISHICHANGDIAOYOU

旧版尊龙人生就是博下载26.6 SparkXIADEJVMNEICUNJIANGDICacheNEICUNZHANBIDEDIAOYOU

DI27ZHANG SparkWUDAZIKUANGJIADIAOYOUZUIJIASHIJIAN

27.1 Spark SQLDIAOYOUYUANLIJIDIAOYOUZUIJIASHIJIAN

旧版尊龙人生就是博下载27.2 Spark StreamingDIAOYOUYUANLIJIDIAOYOUZUIJIASHIJIAN

27.3 Spark GraphXDIAOYOUYUANLIJIDIAOYOUZUIJIASHIJIAN

旧版尊龙人生就是博下载27.4 Spark MLDIAOYOUYUANLIJIDIAOYOUZUIJIASHIJIAN

27.5 SparkRDIAOYOUYUANLIJIDIAOYOUZUIJIASHIJIAN

DI28ZHANG Spark 2.2.0XINYIDAIWUSIJIHUAYOUHUAYINQING

28.1 GAISHU

旧版尊龙人生就是博下载28.2 NEICUNGUANLIYUERJINZHICHULI

旧版尊龙人生就是博下载28.3 HUANCUNGANZHIJISUAN

28.4 DAIMASHENGCHENG

28.5 BENZHANGZONGJIE

旧版尊龙人生就是博下载DI29ZHANG Spark ShuffleDIAOYOUYUANLIJISHIJIAN

29.1 ShuffleDUIXINGNENGXIAOHAODEYUANLIXIANGJIE

29.2 Spark.Shuffle.managerCANSHUDIAOYOUYUANLIJISHIJIAN

29.3 Spark.Shuffle.blockTransferServiceCANSHUDIAOYOUYUANLIJISHIJIAN

旧版尊龙人生就是博下载29.4 Spark.Shuffle.compressCANSHUDIAOYOUYUANLIJISHIJIAN

29.5 Spark.io.compression.codecCANSHUDIAOYOUYUANLIJISHIJIAN

旧版尊龙人生就是博下载29.6 Spark.Shuffle.consolidateFilesCANSHUDIAOYOUYUANLIJISHIJIAN

29.7 Spark.Shuffle.file.bufferCANSHUDIAOYOUYUANLIJISHIJIAN

29.8 Spark.Shuffle.io.maxRetriesCANSHUDIAOYOUYUANLIJISHIJIAN

29.9 Spark.Shuffle.io.retryWaitCANSHUDIAOYOUYUANLIJISHIJIAN

29.10 Spark.Shuffle.io.numConnectionsPerPeerCANSHUDIAOYOUYUANLIJISHIJIAN

29.11 Spark.reducer.maxSizeInFlightCANSHUDIAOYOUYUANLIJISHIJIAN

29.12 Spark.Shuffle.io.preferDirectBufsCANSHUDIAOYOUYUANLIJISHIJIAN

29.13 Spark.Shuffle.memoryFractionCANSHUDIAOYOUYUANLIJISHIJIAN

旧版尊龙人生就是博下载29.14 Spark.Shuffle.service.enabledCANSHUDIAOYOUYUANLIJISHIJIAN

29.15 Spark.Shuffle.service.portCANSHUDIAOYOUYUANLIJISHIJIAN

29.16 Spark.Shuffle.Sort.bypassMergeThresholdCANSHUDIAOYOUYUANLIJISHIJIAN

旧版尊龙人生就是博下载29.17 Spark.Shuffle.spillCANSHUDIAOYOUYUANLIJISHIJIAN

29.18 Spark.Shuffle.spill.compressCANSHUDIAOYOUYUANLIJISHIJIAN

旧版尊龙人生就是博下载DI30ZHANG SparkXINGNENGDIAOYOUZHISHUJUQINGXIEDIAOYOUYIZHANSHIJIEJUEFANGANYUANLIYUSHIZHAN

旧版尊龙人生就是博下载30.1 WEISHENMESHUJUQINGXIESHIFENBUSHIDASHUJUXITONGDEXINGNENGEMENG

30.2 SHUJUQINGXIEJIEJUEFANGANZHIYI:DUIYUANSHUJUJINXINGJUHEBINGGUOLVDIAODAOZHIQINGXIEDEKeys

30.3 SHUJUQINGXIEJIEJUEFANGANZHIER:SHIDANGTIGAOReducerDUANDEBINGXINGDU

旧版尊龙人生就是博下载30.4 SHUJUQINGXIEJIEJUEFANGANZHISAN:SHIYONGSUIJIKeySHIXIANSHUANGZHONGJUHE

30.5 SHUJUQINGXIEJIEJUEFANGANZHISI:SHIYONGMapperDUANJINXINGJoinCAOZUO

30.6 SHUJUQINGXIEJIEJUEFANGANZHIWU:DUIQINGXIEDEKeysCAIYANGHOUJINXINGDANDUDEJoinCAOZUO

旧版尊龙人生就是博下载30.7 SHUJUQINGXIEJIEJUEFANGANZHILIU:SHIYONGSUIJISHUJINXINGJoin

旧版尊龙人生就是博下载30.8 SHUJUQINGXIEJIEJUEFANGANZHIQI:TONGGUOKUORONGJINXINGJoin

旧版尊龙人生就是博下载30.9 JIEHEDIANYINGDIANPINGXITONGJINXINGSHUJUQINGXIEJIEJUEFANGANDEXIAOJIE

DI31ZHANG SparkDASHUJUXINGNENGDIAOYOUSHIZHANZHUANYEZHILU

旧版尊龙人生就是博下载31.1 DASHUJUXINGNENGDIAOYOUDEBENZHIHESparkXINGNENGDIAOYOUYAODIANFENXI

旧版尊龙人生就是博下载31.2 SparkXINGNENGDIAOYOUZHIXITONGZIYUANSHIYONGYUANLIHEDIAOYOUZUIJIASHIJIAN

旧版尊龙人生就是博下载31.3 SparkXINGNENGDIAOYOUZHISHIYONGGENGGAOXINGNENGSUANZIJIQIYUANMAPOUXI

31.4 SparkJIUBANBENZHONGXINGNENGDIAOYOUZHIHashShufflePOUXIJIDIAOYOU

旧版尊龙人生就是博下载31.5 ShuffleRUHECHENGWEISparkXINGNENGSHASHOU

旧版尊龙人生就是博下载31.6 Spark Hash ShuffleYUANMAJIEDUYUPOUXI

旧版尊龙人生就是博下载31.7 Sort-Based ShuffleCHANSHENGDENEIMUJIQItungsten-sortBEIJINGJIEMI

31.8 Spark ShuffleLINGRENFEIJIEDE6DAJINGDIANWENTI

旧版尊龙人生就是博下载31.9 Spark Sort-Based ShufflePAIXUJUTISHIXIANNEIMUHEYUANMAXIANGJIE

31.10 Spark 1.6.XYIQIANShuffleZHONGJVMNEICUNSHIYONGJIPEIZHINEIMUXIANGQING

31.11 Spark 2.2.XZHONGShuffleZHONGNEICUNGUANLIYUANMAJIEMI:StaticMemoryHEUnifiedMemory

旧版尊龙人生就是博下载31.12 Spark 2.2.XZHONGShuffleZHONGJVM Unified MemoryNEIMUXIANGQING

31.13 Spark 2.2.XZHONGShuffleZHONGTaskSHIJIAONEICUNFENPEIGUANLI

旧版尊龙人生就是博下载31.14 Spark 2.2.XZHONGShuffleZHONGMapperDUANDEYUANMASHIXIAN

31.15 Spark 2.2.XZHONGShuffleZHONGSortShuffleWriterPAIXUYUANMANEIMUJIEMI

旧版尊龙人生就是博下载31.16 Spark 2.2.XZHONGSort ShuffleZHONGtimSortPAIXUYUANMAJUTISHIXIAN

31.17 Spark 2.2.XZHONGSort ShuffleZHONGReducerDUANDEYUANMANEIMU

HOUJI

Spark大數據商業實戰三部曲 內核解密 商業案例 性能調優 精彩文摘

DI5ZHANG SparkJIQUNQIDONGYUANLIHEYUANMAXIANGJIE

旧版尊龙人生就是博下载BENZHANGSHENRUJIANGJIESparkJIQUNQIDONGYUANLIHEYUANMA。5.1JIEJIANGJIEMasterQIDONGYUANLIHEYUANMA;5.2JIEJIANGJIEWorkerQIDONGYUANLIHEYUANMA;5.3JIECHANSHULEExecutorBackendQIDONGYUANLIHEYUANMA、ExecutorBackendJIEKOUYUExecutorDEGUANXI、ExecutorBackendDEBUTONGSHIXIAN、ExecutorBackendZHONGDETONGXINJIYICHANGCHULI;5.4JIEJIANGJIEExecutorZHONGRENWUDEZHIXING、JIAZAI、RENWUXIANCHENGCHI、RENWUZHIXINGSHIBAICHULI、TaskRunnerYUNXINGNEIMU;5.5JIEJIANGJIEExecutorZHIXINGJIEGUODECHULIFANGSHI。

5.1 MasterQIDONGYUANLIHEYUANMAXIANGJIE

BENJIEJIANGJIEMasterQIDONGDEYUANLIHEYUANMA;Master HASHUANGJIQIEHUAN;MasterDEZHUCEJIZHIHEZHUANGTAIGUANLIJIEMIDENGNEIRONG。

5.1.1 MasterQIDONGDEYUANLIXIANGJIE

旧版尊龙人生就是博下载SparkYINGYONGCHENGXUZUOWEIDULIDEJIQUNJINCHENGYUNXING,YOUZHUCHENGXUZHONGDESparkContext DUIXIANG(CHENGWEIQUDONGCHENGXU)XIEDIAO。SparkJIQUNBUSHUZUJIANTU5-1SUOSHI。

旧版尊龙人生就是博下载TU5-1 SparkJIQUNBUSHUZUJIANTU

QIZHONGGEGESHUYUJIXIANGGUANSHUYUDEMIAOSHURUXIA。

(1)Driver Program:YUNXINGApplicationDEmainHANSHUBINGXINJIANSparkContextSHILIDECHENGXU,CHENGWEIQUDONGCHENGXU(Driver Program)。TONGCHANGKEYISHIYONGSparkContextDAIBIAOQUDONGCHENGXU。

(2)Cluster Manager:JIQUNGUANLIQI(Cluster Manager)SHIJIQUNZIYUANGUANLIDEWAIBUFUWU。SparkSHANGXIANZAIZHUYAOYOUStandalone、YARN、Mesos 3ZHONGJIQUNZIYUANGUANLIQI。SparkZIDAIDEStandaloneMOSHINENGGOUMANZUJUEDABUFENCHUNCUIDESparkJISUANHUANJINGZHONGDUIJIQUNZIYUANGUANLIDEXUQIU,JIBENSHANGZHIYOUZAIJIQUNZHONGYUNXINGDUOTAOJISUANKUANGJIADESHIHOUCAIJIANYIKAOLVYARNHEMesos。

(3)Worker Node:JIQUNZHONGKEYIYUNXINGApplicationDAIMADEGONGZUOJIEDIAN(Worker Node),XIANGDANGYUHadoopDESlaveJIEDIAN。

(4)Executor:ZAIWorker NodeSHANGWEIApplicationQIDONGDEYIGEGONGZUOJINCHENG,ZAIJINCHENGZHONGFUZERENWU(Task)DEYUNXING,BINGQIEFUZEJIANGSHUJUCUNFANGZAINEICUNHUOCIPANSHANG,ZAIExecutorNEIBUTONGGUODUOXIANCHENGDEFANGSHI(JIXIANCHENGCHI)BINGFACHULIYINGYONGCHENGXUDEJUTIRENWU。

MEIGEApplicationDOUYOUGEZIDULIDEExecutors,YINCIYINGYONGCHENGXUZHIJIANSHIXIANGHUGELIDE。

旧版尊龙人生就是博下载(5)Task:RENWU(Task)SHIZHIBEIDriverSONGDAOExecutorSHANGDEGONGZUODANYUAN。TONGCHANG,YIGERENWUHUICHULIYIGEPartitionDESHUJU,MEIGEPartitionYIBANSHIYIGEHDFSDEBlockKUAIDEDAXIAO。

(6)Application:SHICHUANGJIANLESparkContextSHILIDUIXIANGDESparkYONGHUCHENGXU,BAOHANLEYIGEDriver programHEJIQUNZHONGDUOGEWorkerSHANGDEExecutor。

(7)Job:HESparkDEactionDUIYING,MEIGEaction,RUcount、savaAsTextFileDENGDOUHUIDUIYINGYIGEJobSHILI,MEIGEJobHUICHAIFENCHENGDUOGEStages,YIGEStageZHONGBAOHANYIGERENWUJI(TaskSet),RENWUJIZHONGDEGEGERENWUTONGGUOYIDINGDEDIAODUJIZHIFASONGDAOGONGZUODANWEI(Executor)SHANGBINGXINGZHIXING。

Spark StandaloneJIQUNDEBUSHUCAIYONGDIANXINGDEMaster/SlaveJIAGOU。QIZHONG,MasterJIEDIANFUZEZHENGGEJIQUNDEZIYUANGUANLIYUDIAODU,WorkerJIEDIAN(YEKEYICHENGSlaveJIEDIAN)ZAIMasterJIEDIANDEDIAODUXIAQIDONGExecutor,FUZEZHIXINGJUTIGONGZUO(BAOKUOYINGYONGCHENGXUYIJIYINGYONGCHENGXUTIJIAODERENWU)。

5.1.2 MasterQIDONGDEYUANMAXIANGJIE

SparkZHONGGEGEZUJIANSHITONGGUOJIAOBENLAIQIDONGBUSHUDE。XIAMIANYIJIAOBENWEIRUKOUDIANKAISHIFENXIMasterDEBUSHU。MEIGEZUJIANDUIYINGTIGONGLEQIDONGDEJIAOBEN,TONGSHIYEHUITIGONGTINGZHIDEJIAOBEN。TINGZHIJIAOBENBIJIAOJIANDAN,ZAICIJINFENXIQIDONGJIAOBEN。

圖書網:Spark大數據商業實戰三部曲 內核解密 商業案例 性能調優pdf

繼續閱讀
  • 我的微信
  • 掃一掃加好友
  • weinxin
  • 微信公眾號
  • 掃一掃關注
  • weinxin

有任何需求請前往→→→社區

MySQL王者晉級之路pdf 數據庫

MySQL王者晉級之路pdf

適讀人群 :本書適合熟悉Linux系統且想提升MySQL水平的讀者。 資深數據庫專家十年沉淀 詳細解析MySQL數據庫體系結構 技術問答,助力面試
Spark SQL入門與實踐指南epub 數據庫

Spark SQL入門與實踐指南epub

適讀人群 :Spark初學者、Spark數據分析人員以及Spark程序開發人員 (1)講解Spark SQL背景知識、編程基礎 (2)通過一個工程實例讓讀者領略Spark SQL強...
Oracle數據庫技術與應用pdf 數據庫

Oracle數據庫技術與應用pdf

Oracle數據庫技術與應用 作 者 :天津濱海迅騰科技集團有限公司 Oracle數據庫技術與應用 出版社: 南開大學出版社 Oracle數據庫技術與應用 內容簡介 本書對Oracle數據庫做了全面的...
匿名

發表評論

匿名網友 填寫信息

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen: