I am an undergraduate student and I have to create a Semantic Network based on Wikipedia. This Semantic Network would be similar to Wordnet(except for it is based on Wikipedia and is concerned with "streams of text/topics" rather than simple words etc.) and I am thinking of using the Wikipedia XML dumps for the purpose.

I guess I need to learn parsing an XML and "some other things" related to NLP and probably Machine Learning, but I am no way sure about anything involved herein after the XML parsing.

  • Is the starting step: XML dump parsing into text a good idea/step? Any alternatives?
  • What would be the steps involved after parsing XML into text to create a functional Semantic Network?
  • What are the things/concepts I should learn in order to do them?
  • I am not directly asking for book recommendations, but if you have read a book/article that teaches any thing related/helpful, please mention them. This may include a refernce to already existing implementations regarding the subject.

Please correct me if I was wrong somewhere. Thanks!

EDIT: The final product should be like a complete Semantic Network (like Conceptnet or Cyc etc.), So I can't use things like Semantic Mediawiki.

(On a second thought, it seems like I should have asked this question on linguistics and not here... )

  • what is wordnet??
    – nischayn22
    Jun 25 '12 at 19:40
  • @nischayn22: Wordnet is an example of a Semantic Network. Simply putting, it links different words using relations (like synonyms etc.) to form a network of lexical relations between the vocabulary, and can be used both as dictionary, thesaurus and also for various AI applications. You can read about it here: wordnet.princeton.edu and here: en.wikipedia.org/wiki/WordNet Jun 26 '12 at 14:34

Wikipedia is great for human processing, not so much for machine processing. Have a look at http://wiki.dbpedia.org to get the data extracted from Wikipedia in a machine processable format. Their data set uses OWL as a knowledge representation to provide semantic context to the Wikipedia data set through RDF (XML based) metadata.

Once you have the data set, you will need to be able to run queries against your data. SPARQL is particularly useful in querying data exposed in this format. Consider how you define a stream of data, how you implement your data structures, and how you implement your algorithms that query your data to connect it all.

You can check dbpedia tagged questions for additional guidance.



  • Your answer made me re-think of Dbpedia as an option... I was reluctant to use Dbpedia for the purpose, but since you mentioned it, it does seem like a better option than Wikipedia. I will take a look at DBpedia like you said... Jun 28 '12 at 21:52

Since you mentioned semantic network, you might use semantic mediawiki, I think the process will be to download the database dumps of Wikipedia and adding semantic properties wherever needed. Also are you aware of the Wikidata project??

  • I knew about Semantic Mediawiki before, but I don't think using it for this purpose would be a good idea. The main purpose here is to create a Semantic network based on Wikipedia... I think it was my mistake that I gave the example of Wordnet over there, but I guess I should have mentioned something else. I will edit the question a little now... Jun 26 '12 at 14:25
  • I really appreciate that you wanted to help... Answering the question you asked above: No, I wasn't aware of the Wikidata project! But, that doesn't seem to help me in this situation dude... Jun 28 '12 at 21:54
  • @ForbiddenOverseer for more help you really have to ask the mediawiki guys on irc
    – nischayn22
    Jun 29 '12 at 5:38

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.