movie review dataset

Open Access

Imdb movie reviews dataset.

Abstract 

This dataset contains nearly 1 Million unique movie reviews from 1150 different IMDb movies spread across 17 IMDb genres - Action, Adventure, Animation, Biography, Comedy, Crime, Drama, Fantasy, History, Horror, Music, Mystery, Romance, Sci-Fi, Sport, Thriller and War. The dataset also contains movie metadata such as date of release of the movie, run length, IMDb rating, movie rating (PG-13, R, etc), number of IMDb raters, and number of reviews per movie.

Any feedback on the dataset is welcome.

• Log in to post comments

Why does the file has some weird names??

Can you recheck ... some other files had been uploaded by mistake

The dataset seems to be wrong. It's only 1 MB, and has text files of another language (I think Urdu). Please look into it. Thanks!

Can you check again ... I was experimenting with another dataset ... got uploaded by mistake

Useful dataset with lots of metadata, nice!

Do you know if they include the reviews from the Stanford IMDb large dataset for sentiment classification?

Is there a csv file available that include all movie's genre composition?

Dataset Files

Related datasets, imdb users' ratings dataset, dataset access, how to access this dataset.

This Open Access dataset is available to all IEEE DataPort users. Please login or register.

Login   Create a FREE IEEE Account

Upload your Dataset

How to upload dataset files directly to aws.

IEEE DataPort Subscribers may upload their dataset files directly to IEEE DataPort's AWS S3 file storage. Please read the Upload Your Files directly to the IEEE DataPort S3 Bucket help topic for detailed instructions.

You will need the following information to complete your upload:

• Your AWS Access Key and Secret key, which can be found on your IEEE DataPort User Profile .
• DATASET TYPE: open
• DATASETID: 2565

Dataset Citation

Share / embed, embed this dataset on another website.

Copy and paste the HTML code below to embed your dataset:

Share via email or social media

Click the buttons below:

Share a link to this dataset

Permalink: http://ieee-dataport.org/open-access/imdb-movie-reviews-dataset

DOI Link: https://dx.doi.org/10.21227/zm1y-b270

Short Link: http://ieee-dataport.org/2565

Access on AWS

Want to access the data files.

Open Access data files are available to all users upon login. Login or create a free account today.

Datasets: cornell-movie-review-data / rotten_tomatoes like 53

Dataset Card for "rotten_tomatoes"

Dataset summary.

Movie Review Dataset. This is a dataset of containing 5,331 positive and 5,331 negative processed sentences from Rotten Tomatoes movie reviews. This data was first used in Bo Pang and Lillian Lee, ``Seeing stars: Exploiting class relationships for sentiment categorization with respect to rating scales.'', Proceedings of the ACL, 2005.

Supported Tasks and Leaderboards

More Information Needed

Dataset Structure

Data instances.

• Size of downloaded dataset files: 0.49 MB
• Size of the generated dataset: 1.34 MB
• Total amount of disk used: 1.84 MB

An example of 'validation' looks as follows.

Data Fields

The data fields are the same among all splits.

• text : a string feature.
• label : a classification label, with possible values including neg (0), pos (1).

Data Splits

Reads Rotten Tomatoes sentences and splits into 80% train, 10% validation, and 10% test, as is the practice set out in

Jinfeng Li, ``TEXTBUGGER: Generating Adversarial Text Against Real-world Applications.''

Dataset Creation

Curation rationale, source data, initial data collection and normalization, who are the source language producers, annotations, annotation process, who are the annotators, personal and sensitive information, considerations for using the data, social impact of dataset, discussion of biases, other known limitations, additional information, dataset curators, licensing information, citation information, contributions.

Thanks to @thomwolf , @jxmorris12 for adding this dataset.

Models trained or fine-tuned on cornell-movie-review-data/rotten_tomatoes

tasksource/deberta-small-long-nli

Tasksource/deberta-base-long-nli, sileod/deberta-v3-base-tasksource-nli, sileod/deberta-v3-large-tasksource-nli, xianzhew/distilbert-base-uncased_rotten_tomatoes.

DILAB-HYU/SentiCSE

IMDB Large Movie Review Dataset

The core dataset contains 50,000 reviews split evenly into 25k train and 25k test sets. The overall distribution of labels is balanced (25k pos and 25k neg).

http://ai.stanford.edu/~amaas/data/sentiment/

Character, path to directory where data will be stored. If NULL , user_cache_dir will be used to determine path.

Character. Return training ("train") data or testing ("test") data. Defaults to "train".

Logical, set TRUE to delete dataset.

Logical, set TRUE to return the path of the dataset.

Logical, set TRUE to remove intermediate files. This can greatly reduce the size. Defaults to FALSE.

Logical, set TRUE if you have manually downloaded the file and placed it in the folder designated by running this function with return_path = TRUE .

A tibble with 25,000 rows and 2 variables:

Character, denoting the sentiment

Character, text of the review

In the entire collection, no more than 30 reviews are allowed for any given movie because reviews for the same movie tend to have correlated ratings. Further, the train and test sets contain a disjoint set of movies, so no significant performance is obtained by memorizing movie-unique terms and their associated with observed labels. In the labeled train/test sets, a negative review has a score <= 4 out of 10, and a positive review has a score >= 7 out of 10. Thus reviews with more neutral ratings are not included in the train/test sets. In the unsupervised set, reviews of any rating are included and there are an even number of reviews > 5 and <= 5.

When using this dataset, please cite the ACL 2011 paper

InProceedings{maas-EtAl:2011:ACL-HLT2011, author = {Maas, Andrew L. and Daly, Raymond E. and Pham, Peter T. and Huang, Dan and Ng, Andrew Y. and Potts, Christopher}, title = {Learning Word Vectors for Sentiment Analysis}, booktitle = {Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies}, month = {June}, year = {2011}, address = {Portland, Oregon, USA}, publisher = {Association for Computational Linguistics}, pages = {142--150}, url = {http://www.aclweb.org/anthology/P11-1015} }

IMDB movie review sentiment classification dataset

Load_data function.

Loads the IMDB dataset .

This is a dataset of 25,000 movies reviews from IMDB, labeled by sentiment (positive/negative). Reviews have been preprocessed, and each review is encoded as a list of word indexes (integers). For convenience, words are indexed by overall frequency in the dataset, so that for instance the integer "3" encodes the 3rd most frequent word in the data. This allows for quick filtering operations such as: "only consider the top 10,000 most common words, but eliminate the top 20 most common words".

As a convention, "0" does not stand for a specific word, but instead is used to encode the pad token.

• path : where to cache the data (relative to ~/.keras/dataset ).
• num_words : integer or None. Words are ranked by how often they occur (in the training set) and only the num_words most frequent words are kept. Any less frequent word will appear as oov_char value in the sequence data. If None, all words are kept. Defaults to None .
• skip_top : skip the top N most frequently occurring words (which may not be informative). These words will appear as oov_char value in the dataset. When 0, no words are skipped. Defaults to 0 .
• maxlen : int or None. Maximum sequence length. Any longer sequence will be truncated. None, means no truncation. Defaults to None .
• seed : int. Seed for reproducible data shuffling.
• start_char : int. The start of a sequence will be marked with this character. 0 is usually the padding character. Defaults to 1 .
• oov_char : int. The out-of-vocabulary character. Words that were cut out because of the num_words or skip_top limits will be replaced with this character.
• index_from : int. Index actual words with this index and higher.
• Tuple of Numpy arrays : (x_train, y_train), (x_test, y_test) .

x_train , x_test : lists of sequences, which are lists of indexes (integers). If the num_words argument was specific, the maximum possible index value is num_words - 1 . If the maxlen argument was specified, the largest possible sequence length is maxlen .

y_train , y_test : lists of integer labels (1 or 0).

Note : The 'out of vocabulary' character is only used for words that were present in the training set but are not included because they're not making the num_words cut here. Words that were not seen in the training set but are in the test set have simply been skipped.

get_word_index function

Retrieves a dict mapping words to their index in the IMDB dataset.

The word index dictionary. Keys are word strings, values are their index.

Navigation Menu

Search code, repositories, users, issues, pull requests..., provide feedback.

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly.

To see all available qualifiers, see our documentation .

• Notifications You must be signed in to change notification settings

imdb_reviews.md

Latest commit, file metadata and controls, imdb_reviews.

• Description :

Large Movie Review Dataset. This is a dataset for binary sentiment classification containing substantially more data than previous benchmark datasets. We provide a set of 25,000 highly polar movie reviews for training, and 25,000 for testing. There is additional unlabeled data for use as well.

Additional Documentation : Explore on Papers With Code north_east

Homepage : http://ai.stanford.edu/~amaas/data/sentiment/

Source code : tfds.datasets.imdb_reviews.Builder

• 1.0.0 (default): New split API ( https://tensorflow.org/datasets/splits )

Download size : 80.23 MiB

Auto-cached ( documentation ): Yes

Supervised keys (See as_supervised doc ): ('text', 'label')

Figure ( tfds.show_examples ): Not supported.

imdb_reviews/plain_text (default config)

Config description : Plain text

Dataset size : 129.83 MiB

Feature structure :

• Feature documentation :

label | ClassLabel | | int64 | text | Text | | string |

• Examples ( tfds.as_dataframe ):

{% framebox %}

Display examples...

const contentPane = document.getElementById('dataframecontent'); try { const response = await fetch(url); // Error response codes don't throw an error, so force an error to show // the error message. if (!response.ok) throw Error(response.statusText);

} catch (e) { contentPane.innerHTML = 'Error loading examples. If the error persist, please open ' + 'a new issue.'; } }); </script>

{% endframebox %}

imdb_reviews/bytes

Config description : Uses byte-level text encoding with tfds.deprecated.text.ByteTextEncoder

Dataset size : 129.88 MiB

label | ClassLabel | | int64 | text | Text | (None,) | int64 |

imdb_reviews/subwords8k

Config description : Uses tfds.deprecated.text.SubwordTextEncoder with 8k vocab size

Dataset size : 54.72 MiB

imdb_reviews/subwords32k

Config description : Uses tfds.deprecated.text.SubwordTextEncoder with 32k vocab size

Dataset size : 50.33 MiB

Movie Review Data

Sentiment polarity datasets.

• polarity dataset v2.0 ( 3.0Mb) (includes README v2.0 ): 1000 positive and 1000 negative processed reviews. Introduced in Pang/Lee ACL 2004. Released June 2004.
• Pool of 27886 unprocessed html files (81.1Mb) from which the polarity dataset v2.0 was derived. (This file is identical to movie.zip from data release v1.0.)
• sentence polarity dataset v1.0 (includes sentence polarity dataset README v1.0 : 5331 positive and 5331 negative processed sentences / snippets. Introduced in Pang/Lee ACL 2005. Released July 2005.
• polarity dataset v1.1 (2.2Mb) (includes README.1.1 ): approximately 700 positive and 700 negative processed reviews. Released November 2002. This alternative version was created by Nathan Treloar , who removed a few non-English/incomplete reviews and changing some of the labels (judging some polarities to be different from the original author's rating). The complete list of changes made to v1.1 can be found in diff.txt .
• polarity dataset v0.9 (2.8Mb) (includes a README ):. 700 positive and 700 negative processed reviews. Introduced in Pang/Lee/Vaithyanathan EMNLP 2002. Released July 2002. Please read the "Rating Information - WARNING" section of the README.
• movie.zip (81.1Mb) : all html files we collected from the IMDb archive.

Sentiment scale datasets

• Sep 30, 2009: Yanir Seroussi points out that due to some misformatting in the raw html files, six reviews are misattributed to Dennis Schwartz (29411 should be Max Messier, 29412 should be Norm Schrager, 29418 should be Steve Rhodes, 29419 should be Blake French, 29420 should be Pete Croatto, 29422 should be Rachel Gordon) and one (23982) is blank.

Subjectivity datasets

• subjectivity dataset v1.0 (508K) (includes subjectivity README v1.0 ): 5000 subjective and 5000 objective processed sentences. Introduced in Pang/Lee ACL 2004. Released June 2004.
• Pool of unprocessed source documents (9.3Mb) from which the sentences in the subjectivity dataset v1.0 were extracted. Note : On April 2, 2012, we replaced the original gzipped tarball with one in which the subjective files are now in the correct directory (so that the subjectivity directory is no longer empty; the subjective files were mistakenly placed in the wrong directory, although distinguishable by their different naming scheme).

If you have any questions or comments regarding this site, please send email to Bo Pang or Lillian Lee .

• Tips & How-To
• Newsletters
• White Papers
• .NET Tips and Tricks

The Data Science Lab

• Practical .NET
• The Practical Client
• Data Driver
• PDF Back Issues
• HTML Issue Archive
• Code Samples
• Agile/Scrum
• Open Source
• Cross-Platform C#
• Mobile Corner
• Live! Video

• Visual Studio
• Visual Studio Code
• Blazor/ASP.NET
• C#/VB/TypeScript
• Xamarin/Mobile
• AI/Machine Learning

Preparing IMDB Movie Review Data for NLP Experiments

Dr. James McCaffrey of Microsoft Research shows how to get the raw source IMDB data, read the movie reviews into memory, parse and tokenize the reviews, create a vocabulary dictionary and convert the reviews to a numeric form.

• By James McCaffrey

A common dataset for natural language processing (NLP) experiments is the IMDB movie review data. The goal of an IMDB dataset problem is to predict if a movie review has positive sentiment ("It was a great movie") or negative sentiment ("The film was a waste of time"). A major challenge when working with the IMDB dataset is preparing the data.

This article explains how to get the raw source IMDB data, read the movie reviews into memory, parse and tokenize the reviews, create a vocabulary dictionary and convert the reviews to a numeric form that's suitable for use by a system such as a deep neural network, or an LSTM network, or a Transformer Architecture network.

Most popular neural network libraries, including PyTorch, scikit and Keras, have some form of built-in IMDB dataset designed to work with the library. But there are two problems with using a built-in dataset. First, data access becomes a magic black box and important information is hidden. Second, the built-in datasets use all 25,000 training and 25,000 test movie reviews and these are difficult to work with because they're so large.

A good way to see where this article is headed is to take a look at the screenshot of a Python language program in Figure 1 . The source IMDB movie reviews are stored as text files, one review per file. The program begins by loading all 50,000 movie reviews into memory, and then parsing each review into words/tokens. The words/tokens are used to create a vocabulary dictionary that maps each word/token to an integer ID. For example, the word "the" is mapped to ID = 4.

The vocabulary collection is then used to convert movie reviews that have 20 words or less into token IDs. Reviews that have fewer than 20 words/tokens are padded to exactly length 20 by prepending a special 0 padding ID.

Movie reviews that have positive sentiment, such as, "This is a good movie" get a label of 1 as the last value, and negative sentiment reviews get a label of 0. The result is 16 movie reviews for training and 17 reviews for testing. In a non-demo scenario you'd allow longer reviews, for example, up to 80 words in length, so that you'd get more training and test reviews.

This article assumes you have an intermediate or better familiarity with a C-family programming language, preferably Python, but doesn't assume you know anything about the IMDB dataset. The complete source code for the demo program is presented in this article, and the code is also available in the accompanying file download.

Getting the Source Data Files The IMDB movie review data consists of 50,000 reviews -- 25,000 for training and 25,000 for testing. The training and test files are evenly divided into 12,500 positive reviews and 12,500 negative reviews. Negative reviews are those reviews associated with movies that the reviewer rated as 1 through 4 stars. Positive reviews are the ones rated 7 through 10 stars. Movie reviews that received 5 or 6 stars are considered neither positive nor negative and are not used.

The Large Movie Review Dataset is the primary storage site for the raw IMDB movie reviews data, but you can also find it at other locations using an internet search. If you click on the link on the web page, you will download an 80 MB file in tar-GNU-zip format named aclImdb_v1.tar.gz.

Unlike ordinary .zip compressed files, Windows cannot extract tar.gz files so you need to use an application. I recommend the free 7-Zip utility . After installing 7-Zip you can open Windows File Explorer and then right-click on the aclImdb_v1.tar.gz file and select the Extract Here option. This will result in a 284 MB file named aclImdb_v1.tar ("tape archive"). If you right-click on that tar file and select the Extract Here option, you will get an uncompressed root directory named aclimdb of approximately 300 MB.

The root aclimdb directory contains subdirectories named test and train, plus three files that you can ignore. The test and train directories contain subdirectories named neg and pos, plus five files and one directory named unsup (50,000 unlabeled reviews for unsupervised analysis) that you can ignore. The neg and pos directories each contain 12,500 text files where each review is a single file.

The 50,000 file names look like 102_4.txt where the first part of the file name is the [0] to [12499] review index and the second part of the file name is the numerical review rating (0 to 4 for negative reviews, and 7 to 10 for positive reviews).

The screenshot in Figure 2 shows the directory structure of the IMDB movie review data. The contents of the first positive sentiment training review (file 0.9.txt) is displayed in Notepad.

Making IMDB Reviews Train and Test Files The complete make_data_files.py data preparation program, with a few minor edits to save space, is presented in Listing 1 . The program accepts the 50,000 movie review files as input and creates one training file and one test file.

The program has three helper functions that do all the work:

The get_reviews() function reads all the files in a directory, tokenizes the reviews and returns a list-of-lists such as [["a", "great", "movie"], ["i", "liked", "it", "a", "lot"], . . ["terrific", "film"]]. The make_vocab() function accepts a list of tokenized reviews and builds a dictionary collection where the keys are tokenized words such as "movie" and the values are integer IDs such as 27. The dictionary key-value pairs are also written to a text file named vocab_file.txt so they can be used later by an NLP system.

The generate_file() function accepts the results of get_reviews() and make_vocab() and produces a training or test file. The program control logic is in a main() function which begins:

Next, the vocabulary dictionary is created from the training data:

For the demo, there are 129,888 distinct words/tokens. This is a very large number because in addition to normal English words such as "movie" and "excellent," there are thousands of words specific to movie reviews, such as "hitchcock" (a movie director) and "dicaprio" (an actor).

The vocabulary is based on word frequencies where ID = 4 is the most common word ("the"), ID = 5 is the second most common word ("and") and so on. This allows you to filter out rare words that occur only once or twice.

The vocabulary reserves IDs 0, 1, 2 and 3 for special tokens. ID = 0 is for <PAD> padding. ID = 1 is for <ST> to indicate the start of a sequence. ID = 2 is for <OOV> for out-of-vocabulary words. ID = 3 is reserved but not used. The number of tokens in the entire vocabulary is 129,888 + 4 = 129,892. This number is needed for an embedding layer when creating an NLP prediction system.

The demo program creates a training file with movie reviews that have 20 words or less with these three statements:

The first call to generate_file() uses a "w" argument which creates the destination file for writing the positive reviews. The second call uses a "a" argument to append the negative reviews. It's possible to use "a+" mode but using separate "w" and "a" modes is more clear in my opinion.

The test file is created similarly:

The demo inspects the training file:

The vocabulary dictionary accepts a word/token like "film" and returns an ID like 87. The demo creates a reverse vocabulary object named index_to_word that accepts an ID and returns the corresponding word/token, taking into account the four special tokens:

The demo program concludes by using the modified reverse vocabulary dictionary to decode and display the training file:

There is no standard scheme for NLP vocabulary collections, which is another problem with using built-in IMDB datasets from PyTorch and Keras. Additionally, a vocabulary collection depends entirely upon how the source data is tokenized. This means you must always tokenize NLP data and create an associated vocabulary at the same time.

Listing 1: Program to Create IMDB Movie Review Train and Test Files

• « previous
• next »

Printable Format

.NET MAUI, ASP.NET Core Polished in First Release Candidate for .NET 9

Microsoft shipped the first release candidate for .NET 9, which is nearing feature completeness and production readiness in advance of its November debut.

Microsoft's Stalled UWP Project Supports .NET 9 to Help WinUI 3 Migration

The company urges devs to switch to Windows App SDK and WinUI 3 because UWP is no longer under active development.

Get Up and Running with Modern Angular

The Angular web-dev framework might seem an odd choice for a Microsoft-centric developer to consider, seeing as it's championed by arch-rival Google, but a closer look reveals many advantages.

VS Code Experiments Boost AI Copilot Functionality

Devs can now customize code generation, enjoy enhanced Chat experiences and much more.

• Most Popular Articles
• Most Emailed Articles

Open Source 'Eclipse Theia IDE' Exits Beta to Challenge Visual Studio Code

Adaboost binary classification using c#, asp.net core, .net maui updated as .net 9 nears.

Subscribe on YouTube

Upcoming Training Events

Free Webcasts

• Myths and Realities in Telemetry Data Handling
• How .NET MAUI Changes the Cross-Platform Game Summit
• MoneyTree Achieves Compliance and Speeds Innovation with AWS and Sumo Logic
• Best Practices for AWS Monitoring

> More Webcasts

• Español – América Latina
• Português – Brasil
• Tiếng Việt

Text Classification with Movie Reviews

This notebook classifies movie reviews as positive or negative using the text of the review. This is an example of binary —or two-class—classification, an important and widely applicable kind of machine learning problem.

We'll use the IMDB dataset that contains the text of 50,000 movie reviews from the Internet Movie Database . These are split into 25,000 reviews for training and 25,000 reviews for testing. The training and testing sets are balanced , meaning they contain an equal number of positive and negative reviews.

This notebook uses tf.keras , a high-level API to build and train models in TensorFlow, and TensorFlow Hub , a library and platform for transfer learning. For a more advanced text classification tutorial using tf.keras , see the MLCC Text Classification Guide .

More models

Here you can find more expressive or performant models that you could use to generate the text embedding.

Download the IMDB dataset

The IMDB dataset is available on TensorFlow datasets . The following code downloads the IMDB dataset to your machine (or the colab runtime):

Explore the data

Let's take a moment to understand the format of the data. Each example is a sentence representing the movie review and a corresponding label. The sentence is not preprocessed in any way. The label is an integer value of either 0 or 1, where 0 is a negative review, and 1 is a positive review.

Let's print first 10 examples.

Let's also print the first 10 labels.

Build the model

The neural network is created by stacking layers—this requires three main architectural decisions:

• How to represent the text?
• How many layers to use in the model?
• How many hidden units to use for each layer?

In this example, the input data consists of sentences. The labels to predict are either 0 or 1.

One way to represent the text is to convert sentences into embeddings vectors. We can use a pre-trained text embedding as the first layer, which will have two advantages:

• we don't have to worry about text preprocessing,
• we can benefit from transfer learning.

For this example we will use a model from TensorFlow Hub called google/nnlm-en-dim50/2 .

There are two other models to test for the sake of this tutorial:

• google/nnlm-en-dim50-with-normalization/2 - same as google/nnlm-en-dim50/2 , but with additional text normalization to remove punctuation. This can help to get better coverage of in-vocabulary embeddings for tokens on your input text.
• google/nnlm-en-dim128-with-normalization/2 - A larger model with an embedding dimension of 128 instead of the smaller 50.

Let's first create a Keras layer that uses a TensorFlow Hub model to embed the sentences, and try it out on a couple of input examples. Note that the output shape of the produced embeddings is a expected: (num_examples, embedding_dimension) .

Let's now build the full model:

The layers are stacked sequentially to build the classifier:

• The first layer is a TensorFlow Hub layer. This layer uses a pre-trained Saved Model to map a sentence into its embedding vector. The model that we are using ( google/nnlm-en-dim50/2 ) splits the sentence into tokens, embeds each token and then combines the embedding. The resulting dimensions are: (num_examples, embedding_dimension) .
• This fixed-length output vector is piped through a fully-connected ( Dense ) layer with 16 hidden units.
• The last layer is densely connected with a single output node. This outputs logits: the log-odds of the true class, according to the model.

Hidden units

The above model has two intermediate or "hidden" layers, between the input and output. The number of outputs (units, nodes, or neurons) is the dimension of the representational space for the layer. In other words, the amount of freedom the network is allowed when learning an internal representation.

If a model has more hidden units (a higher-dimensional representation space), and/or more layers, then the network can learn more complex representations. However, it makes the network more computationally expensive and may lead to learning unwanted patterns—patterns that improve performance on training data but not on the test data. This is called overfitting , and we'll explore it later.

Loss function and optimizer

A model needs a loss function and an optimizer for training. Since this is a binary classification problem and the model outputs a probability (a single-unit layer with a sigmoid activation), we'll use the binary_crossentropy loss function.

This isn't the only choice for a loss function, you could, for instance, choose mean_squared_error . But, generally, binary_crossentropy is better for dealing with probabilities—it measures the "distance" between probability distributions, or in our case, between the ground-truth distribution and the predictions.

Later, when we are exploring regression problems (say, to predict the price of a house), we will see how to use another loss function called mean squared error.

Now, configure the model to use an optimizer and a loss function:

Create a validation set

When training, we want to check the accuracy of the model on data it hasn't seen before. Create a validation set by setting apart 10,000 examples from the original training data. (Why not use the testing set now? Our goal is to develop and tune our model using only the training data, then use the test data just once to evaluate our accuracy).

Train the model

Train the model for 40 epochs in mini-batches of 512 samples. This is 40 iterations over all samples in the x_train and y_train tensors. While training, monitor the model's loss and accuracy on the 10,000 samples from the validation set:

Evaluate the model

And let's see how the model performs. Two values will be returned. Loss (a number which represents our error, lower values are better), and accuracy.

This fairly naive approach achieves an accuracy of about 87%. With more advanced approaches, the model should get closer to 95%.

Create a graph of accuracy and loss over time

model.fit() returns a History object that contains a dictionary with everything that happened during training:

There are four entries: one for each monitored metric during training and validation. We can use these to plot the training and validation loss for comparison, as well as the training and validation accuracy:

In this plot, the dots represent the training loss and accuracy, and the solid lines are the validation loss and accuracy.

Notice the training loss decreases with each epoch and the training accuracy increases with each epoch. This is expected when using a gradient descent optimization—it should minimize the desired quantity on every iteration.

This isn't the case for the validation loss and accuracy—they seem to peak after about twenty epochs. This is an example of overfitting: the model performs better on the training data than it does on data it has never seen before. After this point, the model over-optimizes and learns representations specific to the training data that do not generalize to test data.

For this particular case, we could prevent overfitting by simply stopping the training after twenty or so epochs. Later, you'll see how to do this automatically with a callback.

Except as otherwise noted, the content of this page is licensed under the Creative Commons Attribution 4.0 License , and code samples are licensed under the Apache 2.0 License . For details, see the Google Developers Site Policies . Java is a registered trademark of Oracle and/or its affiliates.

Last updated 2023-12-08 UTC.

IMDB Large Movie Review Dataset

Description.

The core dataset contains 50,000 reviews split evenly into 25k train and 25k test sets. The overall distribution of labels is balanced (25k pos and 25k neg).

In the entire collection, no more than 30 reviews are allowed for any given movie because reviews for the same movie tend to have correlated ratings. Further, the train and test sets contain a disjoint set of movies, so no significant performance is obtained by memorizing movie-unique terms and their associated with observed labels. In the labeled train/test sets, a negative review has a score <= 4 out of 10, and a positive review has a score >= 7 out of 10. Thus reviews with more neutral ratings are not included in the train/test sets. In the unsupervised set, reviews of any rating are included and there are an even number of reviews > 5 and <= 5.

When using this dataset, please cite the ACL 2011 paper

InProceedings{maas-EtAl:2011:ACL-HLT2011, author = {Maas, Andrew L. and Daly, Raymond E. and Pham, Peter T. and Huang, Dan and Ng, Andrew Y. and Potts, Christopher}, title = {Learning Word Vectors for Sentiment Analysis}, booktitle = {Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies}, month = {June}, year = {2011}, address = {Portland, Oregon, USA}, publisher = {Association for Computational Linguistics}, pages = {142–150}, url = {http://www.aclweb.org/anthology/P11-1015} }

A tibble with 25,000 rows and 2 variables:

Character, denoting the sentiment

Character, text of the review

http://ai.stanford.edu/~amaas/data/sentiment/

Subscribe to the PwC Newsletter

Join the community, edit dataset, edit dataset tasks.

Some tasks are inferred based on the benchmarks list.

Add a Data Loader

Remove a data loader, edit dataset modalities, edit dataset languages, edit dataset variants.

The benchmarks section lists all benchmarks using a given dataset or any of its variants. We use variants to distinguish between results evaluated on slightly different versions of the same dataset. For example, ImageNet 32⨉32 and ImageNet 64⨉64 are variants of the ImageNet dataset.

Add a new evaluation result row

Movie reviews (movie review polarity dataset enriched with "annotator rationales").

This dataset is based on the movie review polarity dataset (v2.0) collected and maintained by Bo Pang and Lillian Lee. Their dataset (we'll call it PL2.0) consists of 1000 positive and 1000 negative movie reviews obtained from the Internet Movie Database (IMDb) review archive.

The main contribution of this release is the enrichment of the documents with "annotator rationales," a concept we describe in our NAACL HLT 2007 paper.

Basically, "rationales" are segments of the text that support an annotator's classification. Let's say we have a movie review that is labeled as positive (i.e. the writer has a favorable opinion of the movie). Then the rationales would be segments of the text that support the claim (by an annotator) that the review is, indeed, positive.

Here are some examples of positive rationales (the segments enclosed by double square brackets):

• [[you will enjoy the hell out of]] American Pie.
• fortunately, they [[managed to do it in an interesting and funny way]].
• he is [[one of the most exciting martial artists on the big screen]], continuing to perform his own stunts and [[dazzling audiences]] with his flashy kicks and punches.
• the romance was [[enchanting]].

And here are some examples of negative rationales:

• A woman in peril. A confrontation. An explosion. The end. [[Yawn. Yawn. Yawn.]]
• when a film makes watching Eddie Murphy [[a tedious experience, you know something is terribly wrong]].
• the movie is [[so badly put together]] that even the most casual viewer may notice the [[miserable pacing and stray plot threads]].
• [[don't go see]] this movie

Benchmarks Edit Add a new result Link an existing benchmark

Dataset Loaders Edit Add Remove

Similar datasets, tweet sentiment extraction, license edit, modalities edit, languages edit.

Binary Classification of IMDB Movie Reviews

Use keras to classify reviews based on sentiment..

Rakshit Raj

Towards Data Science

Binary Classification refers to classifying samples in one of two categories.

In this example, we will design a neural network to perform two-class classification, or binary classification , of reviews, from the IMDB movie reviews dataset, to determine whether the reviews are positive or negative. We will use the Python library, Keras.

If you are looking for a more fundamental problem, check out solving the MNIST dataset. The content that follows builds primarily on solving MNIST, the ‘hello world! of deep learning.

Solve the MNIST Image Classification Problem

The ‘hello world’ of deep learning and keras in under 10 minutes.

towardsdatascience.com

The IMDB Dataset

The IMDB dataset is a set of 50,000 highly polarized reviews from the Internet Movie Database. They are split into 25000 reviews each for training and testing. Each set contains an equal number (50%) of positive and negative reviews.

The IMDB dataset comes packaged with Keras. It consists of reviews and their corresponding labels (0 for negative and 1 for positive review). The reviews are a sequence of words. They come preprocessed as a sequence of integers, where each integer stands for a specific word in the dictionary.

The IMDB dataset can be loaded directly from Keras and will usually download about 80 MB on your machine.

Loading the Data

Let’s load the prepackaged data from Keras. We will only include 10,000 of the most frequently occurring words.

For kicks, let’s decode the first review.

Preparing the Data

We cannot feed a list of integers into our deep neural network. We will need to convert them into tensors.

To prepare our data, we will One-hot Encode our lists and turn them into vectors of 0’s and 1’s. This would blow up all of our sequences into 10,000-dimensional vectors containing 1 at all indices corresponding to integers present in that sequence. This vector will have element 0 at all index, which is not present in the integer sequence.

Simply put, the 10,000-dimensional vector corresponding to each review will have

• Every index corresponding to a word
• Every index with value 1, is a word that is present in the review and is denoted by its integer counterpart.
• Every index containing 0 is a word not present in the review.

We will vectorize our data manually for maximum clarity. This will result in a tensor of shape (25000, 10000).

Building the Neural Network

Our input data is vectors that need to be mapped to scaler labels (0s and 1s). This is one of the easiest setups, and a simple stack of fully-connected, Dense layers with relu activation perform quite well.

Hidden layers

In this network, we will leverage hidden layers . We will define our layers as such.

The argument being passed to each Dense layer, (16) is the number of hidden units of a layer.

The output from a Dense layer with relu activation is generated after a chain of tensor operations. This chain of operations is implemented as

Where, W is the Weight matrix and b is the bias (tensor).

Having 16 hidden units means that the matrix W will be of the shape ( input_Dimension , 16 ). In this case, where the dimension of the input vector is 10,000, the shape of the Weight matrix will be (10000, 16). If you were to represent this network as a graph, you would see 16 neurons in this hidden layer.

To put it in layman’s terms, there will be 16 balls in this layer.

Each of these balls or hidden units is a dimension in the representation space of the layer. Representation space is the set of all viable representations for the data. Every hidden layer composed of its hidden units aims to learn one specific transformation of the data or one feature/pattern from the data.

DeepAI.org has a very informative write-up on hidden layers.

Hidden layers, simply put, are layers of mathematical functions each designed to produce an output specific to an intended result. Hidden layers allow for the function of a neural network to be broken down into specific transformations of the data. Each hidden layer function is specialized to produce a defined output.For example, a hidden layer functions that are used to identify human eyes and ears may be used in conjunction by subsequent layers to identify faces in images. While the functions to identify eyes alone are not enough to independently recognize objects, they can function jointly within a neural network.

Model Architecture

For our model, we will use

• Two intermediate layers with 16 hidden units each
• Third layer that will output the scalar sentiment prediction
• Intermediate layers will use the relu activation function. relu or Rectified linear unit function will zero out the negative values.
• Sigmoid activation for the final layer or output layer . A sigmoid function “ squashes” arbitrary values into the [0,1] range.

There are formal principles that guide our approach in selecting the architectural attributes of a model. These are not covered in this case study.

Compiling the model

In this step, we will choose an optimizer , a loss function , and metrics to observe. We will go forward with

• binary_crossentropy loss function, commonly used for Binary Classification
• rmsprop optimizer and
• accuracy as a measure of performance

We can pass our choices for optimizer, loss function and metrics as strings to the compile function because rmsprop , binary_crossentropy and accuracy come packaged with Keras.

One could use a customized loss function or optimizer by passing a custom class instance as an argument to the loss , optimizer or mertics fields.

In this example, we will implement our default choices, but we will do so by passing class instances. This is precisely how we would do it if we had customized parameters.

Setting up Validation

We will set aside a part of our training data for validation of the accuracy of the model as it trains. A validation set enables us to monitor the progress of our model on previously unseen data as it goes through epochs during training.

Validation steps help us fine-tune the training parameters of the model.fit function to avoid Overfitting and underfitting of data.

Training our model

Initially, we will train our models for 20 epochs in mini-batches of 512 samples. We will also pass our validation set to the fit method.

Calling the fit method returns a History object. This object contains a member history which stores all data about the training process, including the values of observable or monitored quantities as the epochs proceed. We will save this object to determine the fine-tuning better to apply to the training step.

At the end of the training, we have attained a training accuracy of 99.85% and validation accuracy of 86.57%

Now that we have trained our network, we will observe its performance metrics stored in the History object.

Calling the fit method returns a History object. This object has an attribute history which is a dictionary containing four entries: one per monitored metric.

history_dict contains values of

• Training loss
• Training Accuracy
• Validation Loss
• Validation Accuracy

at the end of each epoch.

Let’s use Matplotlib to plot Training and validation losses and Training and Validation Accuracy side by side.

We observe that minimum validation loss and maximum validation Accuracy is achieved at around 3–5 epochs. After that, we observe two trends:

• increase in validation loss and a decrease in training loss
• decrease in validation accuracy and an increase in training accuracy

This implies that the model is getting better at classifying the sentiment of the training data, but making consistently worse predictions when it encounters new, previously unseen data. This is the hallmark of Overfitting. After the 5th epoch, the model begins to fit too closely to the training data.

To address Overfitting, we will reduce the number of epochs to somewhere between 3 and 5. These results may vary depending on your machine and due to the very nature of the random assignment of weights that may vary from model to model.

In our case, we will stop training after 3 epochs.

Retraining our Neural Network

We retrain our neural network based on our findings from studying the history of loss and accuracy variation. This time we run it for 3 epochs so as to avoid Overfitting on training data.

In the end, we achieve a training accuracy of 99% and a validation accuracy of 86%. This is pretty good, considering we are using a very naive approach. A higher degree of accuracy can be achieved by using a better training algorithm.

Evaluating the model performance

We will use our trained model to make predictions for the test data. The output is an array of floating integers that denote the probability of a review being positive. As you can see, in some cases, the network is absolutely sure the review is positive. In other cases — not so much!

You could try to find some error metric for the number of sentiments that were wrongly classified by using a metric like mean squared error as I did here. But it would be stupid to do so! The analysis of the result is not something we will cover here. However, I will shed some light on why using mse is futile in this case.

The result from our model is the measure of how much the model perceives a review to be positive. Rather than telling us the absolute class of the sample, the model tells us by how much it perceives the sentiment to be skewed on one side or the other. MSE is too simple a metric and fails to capture the complexity of the solution.

I did not visualize this neural net. I would, but it is a time-consuming process. I did visualize the neural network I used in solving the MNIST problem. If you want you could check out this GitHub Project for visualizing ANNs

Prodicode/ann-visualizer

A great visualization python library used to work with keras. it uses python’s graphviz library to create a presentable….

And thus, we have successfully classified reviews on IMDB. I guess this calls for rewatching The Matrix or whatever IMDB suggests next!

I recommend that you work along with the article. You can solve most binary classification problems using a similar strategy. If you did solve it, try fiddling the design and parameters of the network and its layers. This will help you better understand the integrity of the model architecture you chose.

I discuss a single topic in additional detail in each of my articles. In this one, we delved a little bit into hidden layers. An exhaustive explanation of any particular topic is never in the scope of my article; however, you will find ample quick asides.

I assume that the reader has a working understanding of technicalities like an optimizer, categorical encoding, loss function, and metrics. You can find my practice notes on these concepts here .

For more, please check out the book Deep Learning with Python by Francois Chollet .

Feel free to check out this article’s implementation and more of my work on GitHub .

Thanks for reading!

Written by Rakshit Raj

Machine Learning Engineer. Blog: https://www.rakshitraj.com/ LinkedIn: https://www.linkedin.com/in/rakshitraj/

Text to speech

Sentiment Classification on the Large Movie Review Dataset

Data mining project, bert sentiment classification.

• Monticone Pietro
• Moroni Claudio
• Orsenigo Davide

Problem: Sentiment Classification

A sentiment classification problem consists, roughly speaking, in detecting a piece of text and predicting if the author likes or dislikes what he/she is talking about: the input X is a piece of text and the output Y is the sentiment we want to predict, such as the rating of a movie review.

If we can train a model to map X to Y based on a labelled dataset then it can be used to predict sentiment of a reviewer after watching a movie.

Data: Large Movie Review Dataset v1.0

The dataset contains movie reviews along with their associated binary sentiment polarity labels.

• The core dataset contains 50,000 reviews split evenly into 25k train and 25k test sets.
• The overall distribution of labels is balanced (25k pos and 25k neg).
• 50,000 unlabeled documents for unsupervised learning are included, but they won’t be used.
• The train and test sets contain a disjoint set of movies, so no significant performance is obtained by memorizing movie-unique terms and their associated with observed labels.
• In the labeled train/test sets, a negative review has a score ≤ 4 out of 10, and a positive review has a score ≥ 7 out of 10. Thus reviews with more neutral ratings are not included in the train/test sets.
• In the unsupervised set, reviews of any rating are included and there are an even number of reviews > 5 and ≤ 5.

Andrew L. Maas, Raymond E. Daly, Peter T. Pham, Dan Huang, Andrew Y. Ng, and Christopher Potts. (2011). Learning Word Vectors for Sentiment Analysis . The 49th Annual Meeting of the Association for Computational Linguistics (ACL 2011).

Theoretical introduction

The encoder-decoder sequence.

Roughly speaking, an encoder-decoder sequence is an ordered collection of steps ( coders ) designed to automatically translate sentences from a language to another (e.g. the English “the pen is on the table” into the Italian “la penna è sul tavolo”), which could be useful to visualize as follows: input sentence → ( encoders ) → ( decoders ) → output/translated sentence .

For our practical purpose, encoders and decoders are effectively indistinguishable (that’s why we will call them coders ): both are composed of two layers: a LSTM or GRU neural network and an attention module (AM) . They only differ in the way in which their output is processed.

LSTM or GRU neural network

Both the input and the output of an LSTM/GRU neural network consists of two vectors:

• the hidden state : the representation of what the network has learnt about the sentence it’s reading;
• the prediction : the representation of what the network predicts (e.g. translation).

Each word in the English input sentence is translated into its word embedding vector (WEV) before being processed by the first coder (e.g. with word2vec ). The WEV of the first word of the sentence and a random hidden state are processed by the first coder of the sequence. Regarding the output: the prediction is ignored, while the hidden state and the WEV of the second word are passed as input into the second coder and so on to the last word of the sentence. Therefore in this phase the coders work as encoders .

At the end of the sequence of N encoders (N being the number of words in the input sentence), the decoding phase begins:

• the last hidden state and the WEV of the “START” token are passed to the first decoder ;
• the decoder outputs a hidden state and a prection;
• the hidden state and the prediction are passed to the second decoder;
• the second decoder outputs a new hidden state and the second word of the translated/output sentence

and so on up until the whole sentence has been translated, namely when a decoder of the sequence outputs the WEV of the “END” token. Then there is an external mechanism to convert prediction vectors into real words, so it’s very importance to notice that the only purpose of decoders is to predict the next word .

Attention module (AM)

The attention module is a further layer that is placed before the network which provides the collection of words of the sentence with a relational structure. Let’s consider the word “table” in the sentence used as an exampe above. Because of the AM, the encoder will weight the preposition “on” (processed by the previous encoder) more than the article “the” which refers to the subject “cat”.

Bidirectional Encoder Representations from Transformers (BERT)

Transformer.

The transformer is a coder endowed with the AM layer. Transformers have been observed to work much better than the basic encoder-decoder sequences.

BERT is a sequence of encoder-type transformers which was pre-trained to predict a word or sentence (i.e. used as decoder). The benefit of improved performance of Transformers comes at a cost: the loss of bidirectionality , which is the ability to predict both next word and the previous one. BERT is the solution to this problem, a Tranformer which preserves biderectionality .

The first token is not “START”. In order to use BERT as a pre-trained language model for sentence-classification, we need to input the BERT prediction of “CLS” into a linear regression because

• the model has been trained to predict the next sentence, not just the next word;
• the semantic information of the sentence is encoded in the prediction output of “CLS” as a document vector of 512 elements.

• bert_final_data
• https://www.kaggle.com/dataset/5f1193b4685a6e3aa8b72fa3fdc427d18c3568c66734d60cf8f79f2607551a38
• https://www.kaggle.com/dataset/9850d2e4b7d095e2b723457263fbef547437b159e3eb7ed6dc2e88c7869fca0b
• Bert-For-Tf2
• Google github repository
• BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
• A Visual Guide to Using BERT for the First Time
• Machine Translation(Encoder-Decoder Model)!
• The Illustarted Tranformers
• The Illustrated BERT, ELMo, and co. (How NLP Cracked Transfer Learning)
• BERT Explained: State of the art language model for NLP
• Learning Word Vectors for Sentiment Analysis .