MapReduce in Python

Map / Reduce in Python

To understand the concept, please refer to this paper of Google: link

In python 2, map() and reduce() are both built-in functions. However, since Python3, reduce() is no longer a built-in function, one should import it from functools module.
Alt Text

map() applies a function to all the items in an input list and returns an iterator object. Such like:

1
2
3
>> a = map(lambda x:x+x, [1,2,3,4,5])
>> list(a)
Out: [2, 4, 6, 8, 10]

reduce() performs a rolling computation to sequential pairs of values in an input list and returning the result.
Alt Text

1
reduce(f, [x1, x2, x3, x4]) = f(f(f(x1, x2), x3), x4)

Here is an example written by myself today:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
from functools import reduce # only in Python 3

DIGITS = {'0': 0, '1': 1, '2': 2, '3': 3, '4': 4, '5': 5, '6': 6, '7': 7, '8': 8, '9': 9,'.': -1}

def char2num(s):
return DIGITS[s]

def str2float(s):
L = map(char2num, s)
r = reduce(func,L)
print(r)
return r[0]

def func(x, y):

if type(x) == int:
if x == -1:
return (0 + y, 0.1)
elif y == -1:
return(x, 0.1)
else:
return (x*10+y, 1)
elif type(x) == tuple:

if y == -1:
return(x[0], 0.1)
else:
if x[1]< 1:
res = x[1]*0.1
return(x[0]+y*x[1], res)
else:
res = 1
return(x[0]*10+y, res)

if __name__ == "__main__":
#print('str2float(\'123.456\') =', str2float('123.456'))
if abs(str2float('123.4') - 123.4) < 0.00001:
print('测试成功!')
else:
print('测试失败!')

if abs(str2float('123456') - 123456) < 0.00001:
print('测试成功!')
else:
print('测试失败!')

if abs(str2float('0.123456') - 0.123456) < 0.00001:
print('测试成功!')
else:
print('测试失败!')

reference:

  1. https://www.liaoxuefeng.com/wiki/1016959663602400/1017329367486080
  2. https://book.pythontips.com/en/latest/map_filter.html