map()函数

1 map()是python的高阶函数,python高阶函数是指可以把函数作为参数的函数,函数式编程就是指这种高度抽象的编程范式.

要理解高阶函数,首先要明确函数可以赋给变量,函数名本身也是一个变量,也可以给其赋其它值,所以不能随便起变量名,防止与函数名冲突.

map()函数的特殊的地方在它的第一个参数是个指向函数的变量,第二个参数是一个序列,常为list,它将list中的每一个元素输入函数,最后将每个返回值合并成一个新的list返回.

参考：https://www.liaoxuefeng.com/wiki/897692888725344/923030148673312

2 由于map()函数中的第二个参数向第一个参数传参时，是一个一个传，利用这个特点，可是实现字符串的分割.

map()函数在python2中返回的是列表，而在python3中返回的是一个map对象.要想输出显示要加list()

a = 12300000
def shuchu(k):
    return k
print(map(shuchu, str(a)))
b = list(map(shuchu, str(a)))
print(b)
# <map object at 0x7ff47f3c0828>
# ['1', '2', '3', '0', '0', '0', '0', '0']

# 直接用字典对list是无法映射的，下面的写法错误
d = {1:'(', -1:')'}
c = list(map(d, [1,1,-1]))
# TypeError: 'dict' object is not callable

参考：https://www.cnblogs.com/linshuhui/p/8980927.html

3 map函数可以做两个dataframe表格的某些列的融合.

import pandas as pd
import numpy as np

df1 = pd.DataFrame( {'A':[1,2,3,'df2的索引里没有这个,所以融合后是空'],
                     'B':['a','b','c','d'],
                     'C':['Tom','Jack','Bob','roushi']
                   })
print(df1)
df2 = pd.DataFrame( {'A':[1,2,3,4],
                     'B':[6,7,8,9]})
print(df2)
# 相当于从df1的A列与df2的索引融合后,再做映射
df1['df1的A列与df2的索引做融合后再映射'] = df1['A'].map(df2['B'])
print(df1)
#                      A  B       C
# 0                    1  a     Tom
# 1                    2  b    Jack
# 2                    3  c     Bob
# 3  df2的索引里没有这个,所以融合后是空  d  roushi
#    A  B
# 0  1  6
# 1  2  7
# 2  3  8
# 3  4  9
#                      A  B       C  df1的A列与df2的索引做融合后再映射
# 0                    1  a     Tom                   7.0
# 1                    2  b    Jack                   8.0
# 2                    3  c     Bob                   9.0
# 3  df2的索引里没有这个,所以融合后是空  d  roushi                   NaN

4 基本用法

a.字典映射

import pandas as pd
from pandas import Series, DataFrame

data = DataFrame({'food':['bacon','pulled pork','bacon','Pastrami',
   'corned beef','Bacon','pastrami','honey ham','nova lox'],
     'ounces':[4,3,12,6,7.5,8,3,5,6]})
meat_to_animal = {
 'bacon':'pig',
 'pulled pork':'pig',
 'pastrami':'cow',
 'corned beef':'cow',
 'honey ham':'pig',
 'nova lox':'salmon' }
# Python lower() 方法转换字符串中所有大写字符为小写。  因为meat_to_animal中的食物是小写,food列的食物是大写
data['animal'] = data['food'].map(str.lower).map(meat_to_animal)
print(data)
print(data.info())
a = data['food'].map(lambda x: meat_to_animal[x.lower()])
print(a)
#           food  ounces  animal
# 0        bacon     4.0     pig
# 1  pulled pork     3.0     pig
# 2        bacon    12.0     pig
# 3     Pastrami     6.0     cow
# 4  corned beef     7.5     cow
# 5        Bacon     8.0     pig
# 6     pastrami     3.0     cow
# 7    honey ham     5.0     pig
# 8     nova lox     6.0  salmon
# <class 'pandas.core.frame.DataFrame'>
# RangeIndex: 9 entries, 0 to 8
# Data columns (total 3 columns):
# food      9 non-null object
# ounces    9 non-null float64
# animal    9 non-null object
# dtypes: float64(1), object(2)
# memory usage: 296.0+ bytes
# None
# 0       pig
# 1       pig
# 2       pig
# 3       cow
# 4       cow
# 5       pig
# 6       cow
# 7       pig
# 8    salmon
# Name: food, dtype: object

import pandas as pd
df1 = pd.DataFrame({'a':[1,2,3,4,5],
                    'b':['一','二','三','四','五']})
df2 = pd.DataFrame({'c':[5,4,2,1,2,3]})
d = df2['c'].map(dict(zip(df1['a'],df1['b'])))
print(d)
# 0    五
# 1    四
# 2    二
# 3    一
# 4    二
# 5    三
# Name: c, dtype: object

b.与lambda结合使用,函数较简单时,用lambda非常快.

import pandas as pd
from pandas import Series, DataFrame

index = pd.date_range('2017-08-15', periods=10)
ser = Series(list(range(10)), index=index)
print(ser)
ser.index = ser.index.map(lambda x: x.day)
print(ser)
# 2017-08-15    0
# 2017-08-16    1
# 2017-08-17    2
# 2017-08-18    3
# 2017-08-19    4
# 2017-08-20    5
# 2017-08-21    6
# 2017-08-22    7
# 2017-08-23    8
# 2017-08-24    9
# Freq: D, dtype: int64
# 15    0
# 16    1
# 17    2
# 18    3
# 19    4
# 20    5
# 21    6
# 22    7
# 23    8
# 24    9
# dtype: int64

# 实现两个list中元素相乘后再求和
# 注意这里map作用于list时，是将list中的元素一个一个的传过去
a = [1,2,3,4]
b = [2,3,4,5]
sumab = sum(map(lambda x,y:x*y, a,b))
print(sumab)

c.函数映射

d.series映射

参见3

5 Pool和ThreadPool两个模块, 一个基于进程工作, 一个基于线程工作。

使用Pool:

import datetime as dt
import matplotlib.pyplot as plt
import dask.dataframe as dd
from multiprocessing import Pool  

listdata = []
processnum = 12
user_repay = pd.read_hdf('../data/user_repay_second.h5')

for i in range(processnum):
    datai = fenpei(user_repay, i, processnum)
    # print(datai['index'].nunique())# 以index分箱
    listdata.append([i, datai])
del datai
gc.collect()
time1 = dt.datetime.now()
with Pool(processnum) as p:
    p.map(tfun, listdata)
print((dt.datetime.now() - time1).total_seconds())
del listdata
gc.collect()

使用ThreadPool：

import time
from datetime import datetime
from multiprocessing.dummy import Pool as ThreadPool
from functools import partial

def add(x, y):
    print(datetime.now(), "enter add func...")
    time.sleep(2)
    print(datetime.now(), "leave add func...")
    return x+y
def add_wrap(args):
    return add(*args)

if __name__ == "__main__":
    pool = ThreadPool(4) # 池的大小为4
    print(pool.map(add_wrap, [(1,2),(3,4),(5,6)]))
    #close the pool and wait for the worker to exit
    pool.close()
    pool.join()

参考：https://blog.csdn.net/moxiaomomo/article/details/77075125

          https://www.liaoxuefeng.com/wiki/1016959663602400/1017629247922688