完整的中英文词频统计

步骤：

1.准备utf-8编码的文本文件file

2.通过文件读取字符串 str

3.对文本进行预处理

4.分解提取单词 list

5.单词计数字典 set , dict

6.按词频排序 list.sort(key=)

7.排除语法型词汇，代词、冠词、连词等无语义词

8.输出TOP(20)

一、.英文歌曲 词频统计

str2I'm undefeated
Jumpiing out of my skin, pull the chord
Yeah I believe it
The past, is everything we were
don't make us who we are
So I'll dream, until I make it real,
and all I see is stars
Its not until you fall that you fly
When your dreams come alive you're unstoppable
Take a shot, chase the sun, find the beautiful
We will glow in the dark turning dust to gold
And we'll dream it possible
possible
And we'll dream it possible
I will chase, I will reach, I will fly
Until I'm breaking, until I'm breaking
Out of my cage, like a bird in the night
I know I'm changing, I know I'm changing
In, into something big, better than before
And if it takes, takes a thousand lives
Then it's worth fighting for
Its not until you fall that you fly
When your dreams come alive you're unstoppable
Take a shot, chase the sun, find the beautiful
We will glow in the dark turning dust to gold
And we'll dream it possible
it possible
From the bottom to the top
We're sparking wild fire's
Never quit and never stop
The rest of our lives
From the bottom to the top
We're sparking wild fire's
Never quit and never stop
Its not until you fall that you fly
When your dreams come alive you're unstoppable
Take a shot, chase the sun, find the beautiful
We will glow in the dark turning dust to gold
And we'll dream it possible
possible
lower
str2 str2replace
str2 str2replace
str2
str2  str2strip
str2  str2split
str2
 word  str2
   wordstr2countword
strSetstr2
newSet
strSet1strSetnewSet
strSet1
strdict          
 word  strSet1
    strdictword  str2countword
strdictstrdict
strList  strdictitems
 elem
         elem
strListsortkeytakesecondreverse
strList
 i  
     strListi

2.中文小说 词频统计

 jieba
f  encoding
fofread
fclose
fo
doupols  jiebalcutfo
doupodict  
 word  doupols
     word
        doupodictworddoupodictgetword
doupodict
jiebacutfo      
jiebacutfocut_all  
jiebacut_for_searchfo    
wcList  doupodictitems
wcListsortkey   xxreverse   
wcList
 i  
    wcList