0

There is a XML link, which provides the children of any parentID given.

http://www.browsenodes.com/xml.php?action=BrowseNodeInfo&node=1036592

Then you can run the URL again with a children ID and it will provide its children.

When there is no more children of an ID, the return response will have no-children indication.

I created a table in database and inserted ID of top most parent in it. Then i am thinking to do this.

while() {

  1. Select ID from table where deadend = 0
  2. Send it to XML Get its children

If there is NO children found then {

Update this child ID as Deadend

}else{

Insert the child ID in table;

}

But the issue is when the loop will stop ? It will never end this way. How can i make sure that the loop reached to every single child in the tree.

Any clues on this ?

  • What will you do when Select ID from table where deadend = 0 does not return any records? – Dan Pichelman Oct 11 '16 at 20:36
  • i think that means, that the loop reached every thing in the tree so exit (); – brainHax Oct 11 '16 at 20:38
  • That's correct. – Dan Pichelman Oct 11 '16 at 20:43
  • You need to be aware that solutions that involve multiple round trips will not necessarily have atomic/transactional integrity. So, that means you should put in more error checking, and be prepared to encounter the same node id more than once, for example, if the node you'd already visited was moved underneath a part of the tree you hadn't yet visited. – Erik Eidt Oct 11 '16 at 21:43
  • as @Den suggested to mark the ID as 1 after visiting it. And before inserting new ID in the table, we can add an extra check to make sure that this ID does not exist in DB. Wont that help ? – brainHax Oct 11 '16 at 21:54
1

Your algorithm closely resembles Depth First Search

You could implement it this way:

Add the first ID to your database
While (there are unmarked records in the database)
    Get the first ID from the database (mark the record as 'visited')
    Visit the URL, obtain the list of children (if any)
    Add each child to the database *if it is not already there*
    *pause for a while*

Please note this is just a sketch - your actual code will no doubt be much longer and have error checking in it.

I occurred to me that since you're hitting someone's web server for the data, by definition you're implementing a rudimentary web crawler. This isn't necessarily a bad thing, but you should make sure you don't call their server too often (without the pause you might spike their server), honor the site's exclusion policy (robots.txt) and behave responsibly.

  • 1
    actually i am sending my userid in the URL. And adding a 2 second pause in the loop. The XML api on the site is made to be called anyways. They might have implemented some restrictions per userid, as i am already adding my userid in the url . Without userid the API returns few records. – brainHax Oct 11 '16 at 21:48
  • Thanks for the algo anyways, Implemented it as code already :) – brainHax Oct 11 '16 at 21:49

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.