请求URL返回这些数据,Firebug能解析出JSON。但是我用Python的JSON却行不通。我想提取pubkey。
import requests
html = requests.get('https://passport.baidu.com/v2/getpublickey?token=60421c54ebc1272b71e17aa61df10f6a&tpl=tb&apiver=v3&tt=1472291018527&gid=D644D04-1F08-4018-B0E7-6BE79F5C62A6&callback=bd__pcbs__8og8ph', cookies=cookie)
返回:
#字符串(html.text):
bd__pcbs__8og8ph({"errno":'0',"msg":'',"pubkey":'-----BEGIN PUBLIC KEY-----\nMIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQCssku00Ol\/q0t8V8sG\/5oAU76B\n2MXvGWOtmSHDFMK9PwWRLWgTbzynbTbLBvlueRuZX0hhNfGhi1JLrX9lW0HlxWkV\nnSkRAjHAHc7S9JzIEk86+ejWTS0cHHMqhk5\/16d2fuVYXNTDwGD8Hsf62JX2HXhe\nbgoJVhE3ZsVeoyxrowIDAQAB\n-----END PUBLIC KEY-----\n',"key":'QLEP4TeXLB236TtwcSSvHYk0aC24hAWF'})
#二进制(html.content):
b'bd__pcbs__8og8ph({"errno":\'0\',"msg":\'\',"pubkey":\'-----BEGIN PUBLIC KEY-----\\nMIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQDBX1\\/1ZW3WJ7BYeSVJf5TfphiA\\ngBrrQltfCCRD2ZrWsDFaK048+bStUF\\/995PzV3VGsQd\\/gU\\/M\\/jKsxvZrMrFL2cmU\\np2dV8Zffis2vOPSAykC24Pf3ohAN1tbfR2CVGzzp1+gEJPTXW882A+01TFFtgycU\\nrp\\/W3WjIDGFuKKdmGQIDAQAB\\n-----END PUBLIC KEY-----\\n\',"key":\'h14gcxBhwZC6j7bYt4E6l4aU5Hep4POy\'})'
于是我用正则表达式:
pubkey = re.findall(r'-----BEGIN PUBLIC KEY-----.+-----END PUBLIC KEY-----', html.text)[0]
拿到了:
-----BEGIN PUBLIC KEY-----\nMIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQCssku00Ol\/q0t8V8sG\/5oAU76B\n2MXvGWOtmSHDFMK9PwWRLWgTbzynbTbLBvlueRuZX0hhNfGhi1JLrX9lW0HlxWkV\nnSkRAjHAHc7S9JzIEk86+ejWTS0cHHMqhk5\/16d2fuVYXNTDwGD8Hsf62JX2HXhe\nbgoJVhE3ZsVeoyxrowIDAQAB\n-----END PUBLIC KEY-----
然后用来RSA加密,可是RSA要求是b'',我就把pubkey转化成bytes。
pubkey_bytes = bytes(pubkey, encoding='utf-8')
可是里面的\n
、/
转化成了\\n
、\\/
,这样不行啊!
b'-----BEGIN PUBLIC KEY-----\\nMIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQDxhuZTzdfRidIwKBYj8ziTd0rC\\ngNuZFxzdBStw8UYOtD+BKNJ4ccKgrkfAkRW4LlAB8MuNecoW8X0Cb\\/kvauobSa5r\\nM7DwIOQWtN9fCFCIe1Xa9gSdXYrL\\/0X57Dtmw\\/DXmzfMFGsbHnB9zCmgVAM9IGXo\\nZr9skrfXx\\/sRMtBZ5QIDAQAB\\n-----END PUBLIC KEY-----'
RSA公钥必须是这样的格式:
#字符串:
-----BEGIN PUBLIC KEY-----
MIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQCssku00Ol/q0t8V8sG/5oAU76B
2MXvGWOtmSHDFMK9PwWRLWgTbzynbTbLBvlueRuZX0hhNfGhi1JLrX9lW0HlxWkV
nSkRAjHAHc7S9JzIEk86+ejWTS0cHHMqhk5/16d2fuVYXNTDwGD8Hsf62JX2HXhe
bgoJVhE3ZsVeoyxrowIDAQAB
-----END PUBLIC KEY-----
#二进制:
b'-----BEGIN PUBLIC KEY-----\nMIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQDX1kWowXZuatDRaPkjTNFA2hJ7\n857jM8oWgBJmS9JXZdJxFK70Th2cWsL8/cvLDzxs0lBaPtZwK7XvphipFXvCLDCn\naz58KJyrqiDrmNpDMPjIFzf6n+Bk0SQlW4KuNJoy3VS18AlS7v5gV7OvscQHPgNU\nGtUxUPdgKT5zsCT0+QIDAQAB\n-----END PUBLIC KEY-----'
三天两夜了,实在是找不到解决办法了。最后搜肠刮肚用了这么个办法:
re.sub(r'\\/', r'/', re.sub(r'\\n', r'\n', pubkey)) #把pubkey里面的\\n和\/替换成\n和/
有没有更好的办法?
正则可以直接匹配 bytes,不用中间转来转去的~
#二进制(html.content):
content = (
b'bd__pcbs__8og8ph('
b'{"errno":\'0\',"msg":\'\','
b'"pubkey":\'-----BEGIN PUBLIC KEY-----\\n'
b'MIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQDBX1\\/1ZW3WJ7BYeSVJf5TfphiA\\n'
b'gBrrQltfCCRD2ZrWsDFaK048+bStUF\\/995PzV3VGsQd\\/gU\\/M\\/jKsxvZrMrFL2cmU\\n'
b'p2dV8Zffis2vOPSAykC24Pf3ohAN1tbfR2CVGzzp1+gEJPTXW882A+01TFFtgycU\\n'
b'rp\\/W3WjIDGFuKKdmGQIDAQAB\\n'
b'-----END PUBLIC KEY-----\\n\','
b'"key":\'h14gcxBhwZC6j7bYt4E6l4aU5Hep4POy\'})'
)
import re
ptn_pk = re.compile(b'(?<="pubkey":\')[^\']+')
pkb = ptn_pk.findall(content)[0]
pkb = pkb.replace(b'\\n',b'\n').replace(b'\\/',b'/')
print(pkb) # 二进制
print(pkb.decode()) # text
该问题已经解决!附上解决办法:
问题出在无论请求的URL是什么,返回的代码都不是标准的JSON代码。并不是因为被bd__pcbs__8og8ph
包括起来了,也不是去掉bd__pcbs__8og8ph
参数就可以被JSON解析。是因为返回的代码里有单引号的键值对。
通过html.text
拿到字符串,使用正则re.sub
把其中的单引号替换成双引号,然后使用json.loads
声明成JSON对象,最后pubkey_json[pubkey]
拿到其中的pubkey。
具体代码如下:
pubkeyHtml = requests.get('https://passport.baidu.com/v2/getpublickey?token=60421c54ebc1272b71e17aa61df10f6a&tpl=tb&apiver=v3&tt=1472291018527&gid=D644D04-1F08-4018-B0E7-6BE79F5C62A6&ie=utf-8', cookies=cookie) #无论是否带入`bd__pcbs__8og8ph`参数都可以
pubkeyStr = re.sub(r'\'', r'"', pubkeyHtml.text) #替换单引号
pubkey = json.loads(pubkeyStr)['pubkey'] #创建JSON对象并拿到pubkey字符串
print('当前提取到的pubkey是:', pubkey)