training neural network
$begingroup$
I was given the task as follows,
Scrape articles appearing in Times of India since 2010 on HIV and AIDS. Classify them using training a neural network of your choice. Find patterns in those articles, analyze the data.
I have done the scraping part, But I have no clue what to do now with the data I have extracted. I searched for hours on internet but could not find any help.
I know theory part about the neural network but lack knowledge for implementing it on a text. It would be of great help if someone can suggest me any method or provide me some helpful link.
machine-learning neural-network deep-learning classification lstm
edited Dec 25 '18 at 7:17
Media
6,75552057
asked Dec 25 '18 at 7:01
Sandip KumarSandip Kumar
163
$endgroup$
add a comment |
$begingroup$
I was given the task as follows,
Scrape articles appearing in Times of India since 2010 on HIV and AIDS. Classify them using training a neural network of your choice. Find patterns in those articles, analyze the data.
I have done the scraping part, But I have no clue what to do now with the data I have extracted. I searched for hours on internet but could not find any help.
I know theory part about the neural network but lack knowledge for implementing it on a text. It would be of great help if someone can suggest me any method or provide me some helpful link.
machine-learning neural-network deep-learning classification lstm
edited Dec 25 '18 at 7:17
Media
6,75552057
asked Dec 25 '18 at 7:01
Sandip KumarSandip Kumar
163
$endgroup$
add a comment |
$begingroup$
I was given the task as follows,
Scrape articles appearing in Times of India since 2010 on HIV and AIDS. Classify them using training a neural network of your choice. Find patterns in those articles, analyze the data.
I have done the scraping part, But I have no clue what to do now with the data I have extracted. I searched for hours on internet but could not find any help.
I know theory part about the neural network but lack knowledge for implementing it on a text. It would be of great help if someone can suggest me any method or provide me some helpful link.
machine-learning neural-network deep-learning classification lstm
edited Dec 25 '18 at 7:17
Media
6,75552057
asked Dec 25 '18 at 7:01
Sandip KumarSandip Kumar
163
$endgroup$
I was given the task as follows,
Scrape articles appearing in Times of India since 2010 on HIV and AIDS. Classify them using training a neural network of your choice. Find patterns in those articles, analyze the data.
I have done the scraping part, But I have no clue what to do now with the data I have extracted. I searched for hours on internet but could not find any help.
I know theory part about the neural network but lack knowledge for implementing it on a text. It would be of great help if someone can suggest me any method or provide me some helpful link.
machine-learning neural-network deep-learning classification lstm
machine-learning neural-network deep-learning classification lstm
edited Dec 25 '18 at 7:17
Media
6,75552057
asked Dec 25 '18 at 7:01
Sandip KumarSandip Kumar
163
edited Dec 25 '18 at 7:17
Media
6,75552057
asked Dec 25 '18 at 7:01
Sandip KumarSandip Kumar
163
edited Dec 25 '18 at 7:17
Media
6,75552057
edited Dec 25 '18 at 7:17
Media
6,75552057
edited Dec 25 '18 at 7:17
Media
6,75552057
6,75552057
asked Dec 25 '18 at 7:01
Sandip KumarSandip Kumar
163
asked Dec 25 '18 at 7:01
Sandip KumarSandip Kumar
163
asked Dec 25 '18 at 7:01
Sandip KumarSandip Kumar
163
163
add a comment |
add a comment |
2 Answers
2
active
oldest
votes
$begingroup$
There are many solutions for this task. I suggest one of them. As you know, words have relation and if you choose to give each word a special code, you can't have this relation. Consequently, first try to use an embedding network in order to assign each word a code. Then assign each article a label. Next, for each article, you have a sequence of words, codes, which are now embedded. You can employ LSTM networks for classification.
If you are not very familiar with the concepts I referred to, you may want to look for Word2Vec.
answered Dec 25 '18 at 7:16
MediaMedia
6,75552057
$endgroup$
add a comment |
$begingroup$
First of all, can you tell us a bit more about the classification, as in classify the texts into what classes?
Now, to answer your question,
You have input of text sentences which are articles related to HIV/AIDS. Now, you want to extract information from them.
To do this, you'll need a model that "understands" the contextual meaning of the words in the text sentences. Hence, if you start by one-hot encoding the words in your sentences, this model will perform poorly as that encoding will not contain any information about context in the text.
To solve this problem, you'll need Embedding layers.
Embedding layers help in representing words with similar meanings in similar fashion.
Word Embeddings are actually learned from text data. It is very common to see embeddings that are 256 or 512 dimensional. While one hot encoding would result in dimensionality of the size of your word-set, embeddings hold a lot of information in lesser dimension.
There are 2 ways to use them in your model:
- To learn the embeddings while training your model.In this method, you start with random word vectors and learning them as you learn weights of your neural networks.
- Use pre-trained embeddings. These are pre computed embeddings which can be loaded into your model.
Some examples of pre-trained word embeddings include :
-> glove
-> Word2Vec
->Fasttext
Once you've converted the text into their embeddings using any of the above methods, now you can feed them to your neural network (RNN/LSTM/CNN) for your classification task.
Hope this helps :)
answered Dec 28 '18 at 7:49
Mohit BanerjeeMohit Banerjee
1362
$endgroup$
add a comment |
Your Answer
StackExchange.ifUsing("editor", function () {
return StackExchange.using("mathjaxEditing", function () {
StackExchange.MarkdownEditor.creationCallbacks.add(function (editor, postfix) {
StackExchange.mathjaxEditing.prepareWmdForMathJax(editor, postfix, [["$", "$"], ["\\(","\\)"]]);
});
});
}, "mathjax-editing");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "557"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: false,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: null,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fdatascience.stackexchange.com%2fquestions%2f43116%2ftraining-neural-network%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
2 Answers
2
active
oldest
votes
2 Answers
2
active
oldest
votes
active
oldest
votes
active
oldest
votes
$begingroup$
There are many solutions for this task. I suggest one of them. As you know, words have relation and if you choose to give each word a special code, you can't have this relation. Consequently, first try to use an embedding network in order to assign each word a code. Then assign each article a label. Next, for each article, you have a sequence of words, codes, which are now embedded. You can employ LSTM networks for classification.
If you are not very familiar with the concepts I referred to, you may want to look for Word2Vec.
answered Dec 25 '18 at 7:16
MediaMedia
6,75552057
$endgroup$
add a comment |
$begingroup$
There are many solutions for this task. I suggest one of them. As you know, words have relation and if you choose to give each word a special code, you can't have this relation. Consequently, first try to use an embedding network in order to assign each word a code. Then assign each article a label. Next, for each article, you have a sequence of words, codes, which are now embedded. You can employ LSTM networks for classification.
If you are not very familiar with the concepts I referred to, you may want to look for Word2Vec.
answered Dec 25 '18 at 7:16
MediaMedia
6,75552057
$endgroup$
add a comment |
$begingroup$
There are many solutions for this task. I suggest one of them. As you know, words have relation and if you choose to give each word a special code, you can't have this relation. Consequently, first try to use an embedding network in order to assign each word a code. Then assign each article a label. Next, for each article, you have a sequence of words, codes, which are now embedded. You can employ LSTM networks for classification.
If you are not very familiar with the concepts I referred to, you may want to look for Word2Vec.
answered Dec 25 '18 at 7:16
MediaMedia
6,75552057
$endgroup$
There are many solutions for this task. I suggest one of them. As you know, words have relation and if you choose to give each word a special code, you can't have this relation. Consequently, first try to use an embedding network in order to assign each word a code. Then assign each article a label. Next, for each article, you have a sequence of words, codes, which are now embedded. You can employ LSTM networks for classification.
If you are not very familiar with the concepts I referred to, you may want to look for Word2Vec.
answered Dec 25 '18 at 7:16
MediaMedia
6,75552057
edited Dec 25 '18 at 9:20
answered Dec 25 '18 at 7:16
MediaMedia
6,75552057
answered Dec 25 '18 at 7:16
MediaMedia
6,75552057
answered Dec 25 '18 at 7:16
MediaMedia
6,75552057
6,75552057
add a comment |
add a comment |
$begingroup$
First of all, can you tell us a bit more about the classification, as in classify the texts into what classes?
Now, to answer your question,
You have input of text sentences which are articles related to HIV/AIDS. Now, you want to extract information from them.
To do this, you'll need a model that "understands" the contextual meaning of the words in the text sentences. Hence, if you start by one-hot encoding the words in your sentences, this model will perform poorly as that encoding will not contain any information about context in the text.
To solve this problem, you'll need Embedding layers.
Embedding layers help in representing words with similar meanings in similar fashion.
Word Embeddings are actually learned from text data. It is very common to see embeddings that are 256 or 512 dimensional. While one hot encoding would result in dimensionality of the size of your word-set, embeddings hold a lot of information in lesser dimension.
There are 2 ways to use them in your model:
- To learn the embeddings while training your model.In this method, you start with random word vectors and learning them as you learn weights of your neural networks.
- Use pre-trained embeddings. These are pre computed embeddings which can be loaded into your model.
Some examples of pre-trained word embeddings include :
-> glove
-> Word2Vec
->Fasttext
Once you've converted the text into their embeddings using any of the above methods, now you can feed them to your neural network (RNN/LSTM/CNN) for your classification task.
Hope this helps :)
answered Dec 28 '18 at 7:49
Mohit BanerjeeMohit Banerjee
1362
$endgroup$
add a comment |
$begingroup$
First of all, can you tell us a bit more about the classification, as in classify the texts into what classes?
Now, to answer your question,
You have input of text sentences which are articles related to HIV/AIDS. Now, you want to extract information from them.
To do this, you'll need a model that "understands" the contextual meaning of the words in the text sentences. Hence, if you start by one-hot encoding the words in your sentences, this model will perform poorly as that encoding will not contain any information about context in the text.
To solve this problem, you'll need Embedding layers.
Embedding layers help in representing words with similar meanings in similar fashion.
Word Embeddings are actually learned from text data. It is very common to see embeddings that are 256 or 512 dimensional. While one hot encoding would result in dimensionality of the size of your word-set, embeddings hold a lot of information in lesser dimension.
There are 2 ways to use them in your model:
- To learn the embeddings while training your model.In this method, you start with random word vectors and learning them as you learn weights of your neural networks.
- Use pre-trained embeddings. These are pre computed embeddings which can be loaded into your model.
Some examples of pre-trained word embeddings include :
-> glove
-> Word2Vec
->Fasttext
Once you've converted the text into their embeddings using any of the above methods, now you can feed them to your neural network (RNN/LSTM/CNN) for your classification task.
Hope this helps :)
answered Dec 28 '18 at 7:49
Mohit BanerjeeMohit Banerjee
1362
$endgroup$
add a comment |
$begingroup$
First of all, can you tell us a bit more about the classification, as in classify the texts into what classes?
Now, to answer your question,
You have input of text sentences which are articles related to HIV/AIDS. Now, you want to extract information from them.
To do this, you'll need a model that "understands" the contextual meaning of the words in the text sentences. Hence, if you start by one-hot encoding the words in your sentences, this model will perform poorly as that encoding will not contain any information about context in the text.
To solve this problem, you'll need Embedding layers.
Embedding layers help in representing words with similar meanings in similar fashion.
Word Embeddings are actually learned from text data. It is very common to see embeddings that are 256 or 512 dimensional. While one hot encoding would result in dimensionality of the size of your word-set, embeddings hold a lot of information in lesser dimension.
There are 2 ways to use them in your model:
- To learn the embeddings while training your model.In this method, you start with random word vectors and learning them as you learn weights of your neural networks.
- Use pre-trained embeddings. These are pre computed embeddings which can be loaded into your model.
Some examples of pre-trained word embeddings include :
-> glove
-> Word2Vec
->Fasttext
Once you've converted the text into their embeddings using any of the above methods, now you can feed them to your neural network (RNN/LSTM/CNN) for your classification task.
Hope this helps :)
answered Dec 28 '18 at 7:49
Mohit BanerjeeMohit Banerjee
1362
$endgroup$
First of all, can you tell us a bit more about the classification, as in classify the texts into what classes?
Now, to answer your question,
You have input of text sentences which are articles related to HIV/AIDS. Now, you want to extract information from them.
To do this, you'll need a model that "understands" the contextual meaning of the words in the text sentences. Hence, if you start by one-hot encoding the words in your sentences, this model will perform poorly as that encoding will not contain any information about context in the text.
To solve this problem, you'll need Embedding layers.
Embedding layers help in representing words with similar meanings in similar fashion.
Word Embeddings are actually learned from text data. It is very common to see embeddings that are 256 or 512 dimensional. While one hot encoding would result in dimensionality of the size of your word-set, embeddings hold a lot of information in lesser dimension.
There are 2 ways to use them in your model:
- To learn the embeddings while training your model.In this method, you start with random word vectors and learning them as you learn weights of your neural networks.
- Use pre-trained embeddings. These are pre computed embeddings which can be loaded into your model.
Some examples of pre-trained word embeddings include :
-> glove
-> Word2Vec
->Fasttext
Once you've converted the text into their embeddings using any of the above methods, now you can feed them to your neural network (RNN/LSTM/CNN) for your classification task.
Hope this helps :)
answered Dec 28 '18 at 7:49
Mohit BanerjeeMohit Banerjee
1362
answered Dec 28 '18 at 7:49
Mohit BanerjeeMohit Banerjee
1362
answered Dec 28 '18 at 7:49
Mohit BanerjeeMohit Banerjee
1362
answered Dec 28 '18 at 7:49
Mohit BanerjeeMohit Banerjee
1362
1362
add a comment |
add a comment |
Thanks for contributing an answer to Data Science Stack Exchange!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
Use MathJax to format equations. MathJax reference.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fdatascience.stackexchange.com%2fquestions%2f43116%2ftraining-neural-network%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
