NotesWhat is notes.io?

Notes brand slogan

Notes - notes.io

# question 1 - Perform tokenization (Whitespace, Punctuation-based, Treebank, Tweet, MWE)
#using NLTK library. Use porter stemmer and snowball stemmer for stemming. Use
#any technique for lemmatization.
import
import nltk
nltk
from
from nltk.tokenize
nltk.tokenize import
from nltk.stem
nltk.stem import
import WhitespaceTokenizer, WordPunctTokenizer, TreebankWordTokenizer,
TweetTokenizer, MWETokenizer
from
import PorterStemmer, SnowballStemmer
from
from nltk.stem
nltk.stem import
import WordNetLemmatizer
text = "The quick brown fox's jumps were amazing!"
# Tokenizers
print(WhitespaceTokenizer().tokenize(text))
print(WordPunctTokenizer().tokenize(text))
print(TreebankWordTokenizer().tokenize(text))
print(TweetTokenizer().tokenize(text))
mwe = MWETokenizer([('brown', 'fox')])
print(mwe.tokenize(text.lower().split()))
# Stemming
porter = PorterStemmer()
snowball = SnowballStemmer("english")
print([porter.stem(w) for
print([snowball.stem(w) for
for w in
in text.split()])
for w in
in text.split()])
# Lemmatization
nltk.download('wordnet')
lemmatizer = WordNetLemmatizer()
print([lemmatizer.lemmatize(w) for
for w in
in text.split()])
     
 
what is notes.io
 

Notes is a web-based application for online taking notes. You can take your notes and share with others people. If you like taking long notes, notes.io is designed for you. To date, over 8,000,000,000+ notes created and continuing...

With notes.io;

  • * You can take a note from anywhere and any device with internet connection.
  • * You can share the notes in social platforms (YouTube, Facebook, Twitter, instagram etc.).
  • * You can quickly share your contents without website, blog and e-mail.
  • * You don't need to create any Account to share a note. As you wish you can use quick, easy and best shortened notes with sms, websites, e-mail, or messaging services (WhatsApp, iMessage, Telegram, Signal).
  • * Notes.io has fabulous infrastructure design for a short link and allows you to share the note as an easy and understandable link.

Fast: Notes.io is built for speed and performance. You can take a notes quickly and browse your archive.

Easy: Notes.io doesn’t require installation. Just write and share note!

Short: Notes.io’s url just 8 character. You’ll get shorten link of your note when you want to share. (Ex: notes.io/q )

Free: Notes.io works for 14 years and has been free since the day it was started.


You immediately create your first note and start sharing with the ones you wish. If you want to contact us, you can use the following communication channels;


Email: [email protected]

Twitter: http://twitter.com/notesio

Instagram: http://instagram.com/notes.io

Facebook: http://facebook.com/notesio



Regards;
Notes.io Team

     
 
Shortened Note Link
 
 
Looding Image
 
     
 
Long File
 
 

For written notes was greater than 18KB Unable to shorten.

To be smaller than 18KB, please organize your notes, or sign in.