Overview

Your job is to build a system architecture in which agents can migrate from one host process to another, while maintaining their internal state. Users interact via the web with the agents, one user to one agent. Users get data in the form of labels and matching data (think, e.g., quiz questions and answers), and can also submit labels and matching data. Agents belong to a group. Agents share data with members of their group. Maintain a NameServer that knows the location of all agents and HostServers, and provides utility services.

Note: Substantial extra credit for DIA systems that share data with the systems of other students! Make sure I know. Define your interface and protocol! (Hosting of the agents of other students should not be attempted.)

(Note: Old agent program page This has much useful information. In particular you might wish to run the Web animal game, and look over the code, to get a feel for dynamically generated Web pages. The LUNCH PROBLEM discussion is also relevant.)

Submission Files

• Agent Checklist
• NameServer.java
• HostServer.java
• Optional: DIAdb.java <-- possibly needed for single repository of data
• ServerLogs.txt
• DIADiscussion.doc or (DIADiscussion.txt or DIADiscussion.html) No other formats!
• (optional) Readme.txt <-- Special instructions

Basic Grading Procedure

1. Read your checklist to see which features have been implemented
2. We will run the GradeAgent Batch File , under windows, to see your program work and to...
   • Start your NameServer
   • Start your DIAdb if provided
   • Start your first HostServer
   • Start a second HostServer by passing a different port number than the default
   • Connect to the first HostServer three times to create agents one to three
   • Connect to the second HostServer three times to create agents four to six
   • Check the NameServer to see if both HostServers and six agents are registered. See if agents belong to groups.
   • Enter, and retrieve, data for various agents in one group.
   • Check to see if the data is shared for agents in that group.
   • See if agents migrate
   • Make a top-level query of yor agent to dump its state, and verify that the state is consistent with its known history.
3. If you have a more complicated procedure to compile and run your code, then provide a file name, explicitly, rundia.bat that will compile, run, start browsers, for us to see your code work.
4. [Compile your NameServer.java, and HostServer.java code (and DIAdb.java if used).]
5. Read your ServerLogs.txt file (Composite of NameServer and HostServer sample server logs).
6. Read your program code. Read the COMMENTS in your program code.
7. Read your DIADiscussion file. How did you solve the lunch problem? ...share data? ...migrate agents?
8. Follow the checklist to verify all features in the code.
9. Follow your instructions to run the special features of your assignment

What to do first -- recommendations

1. Start your DIADiscussion file early, and keep notes as you develop. Use this file to make your case that you understand the issues. Remember this programming assignment is supplementary to the lectures.
2. Write comments in your code as you go. This is pedagogical code; comments count.
3. Get the basic HostServers running to support agents. Hard-code in the NameServer functions to start
4. Establish simple user communication with your agents - no data posting at first.
5. Hard-code in your agent migration from one HostServer to another with no zombies.
6. If you are going to use zombies, then build them now.
7. Build your NameServer to display dynamic HTML.
8. Integrate your NameServer with your HostServer so that when agents are created, they get humand names from, and are registered by, the NameServer
9. Integrate migration so that the NameServer is updated.
10. On paper, develop your model of group membership. Then implement it with the NameServer and the HostServers.
11. Build the user interface dynamic HTML so that your users can review, and add, data.
12. On paper, develop your model of sharing data. DO THIS FIRST
13. Implement your model of sharing data.

Nameserver

1. Displays all agents and HostServers in the system, retrievable via the web, in a table.
2. When a HostServer is created, notification is sent to the NameServer with the HostServer's ip address and port.
3. When an agent migrates, a notification is sent to the nameserver with the agent's ID number, and new endpoint.
4. When an agent changes groups, a notification is sent to the nameserver with the agent's ID number, and new group.
5. "Clicking on" the agent's displayed link from the nameserver places a web client in direct communication with the agent.
6. (Optional) "Clicking on" the agent's kill link sends a message to the agent to kill itself.
7. In a separate section of the nameserver page, provide hotlinks to all of the active HostServers in your system.
8. "Clicking on" the hotlink for a HostServer puts you in direct communication with that HostServer.
9. (Optional) you might wish to display other information about your HostServers, such as how many agents they are hosting. (But keep it to one line!)
10. Keep a list of at least twenty one-time-use human names, such as Alice, Bob, Carold, Dave, Frank, and so on. When a HostServer connects to register a new agent, provide a unique human name for that agent. (Once used, remove the name from the available list!)
11. Server Loop. Your NameServer must provide some utility functions through a server loop as follows:
    1. When an agent wants to migrate, it can connect to the NameServer and find an available HostServer from the list registered. Assign a HostServer randomly, including, possibly, the same HostServer.
    2. When a HostServer is creating an agent, it must assign the agent to a group. Provide a service so that a HostService can request a human name, the name and ID# of a group, and all the other current members. When this information is requested you must pass back the current IP address and agent communication port of all agents in the group so that the new agent will be able to communicate with them.
    3. When a HostServer is creating an agent, notification is sent to the NameServer with the agent's name, ID number, current user endpoint (ip address and port number), current agent-communication port, and optionally current agent administration port, and they are added to the nameserver table. The NameServer provides a unique human name for the agent (see above), determines a group affiliation (color / group number) and returns this to the HostServer along with the agent-communication endpoints of all other agents in the group so that the new agent will be able to communicate autonomously with its group. It is fine to hard-code three different group ID#s (use ascii string for ease of HTML?), into your NameServer.

Interface

1. All programs use the ports specified.
2. For our testing purposes your agent system should run on localhost. However, you are strongly encouraged to run your agents on multiple machines at home. A simple way to effect this is to pass the IP address of your NameServer as a (second) argument that otherwise defaults to localhost.
3. When starting a HostServer it must use the default HostServer port, but must also accept as the first argument an alternate port.
4. Starting the NameServer means that is that it is ready to register agents and HostServers, to update agent location and group information, to provide human names for new agents, and to provide a random valid HostServer IP address when an agent wants to migrate.
5. Running your HostServer program automatically registers it with the NameServer.
6. When a user makes the default connection to some HostServer, this starts up an agent, and, in turn the HostServer registers that agent with the NameServer. The user is now in direct communication with the agent and is no longer communicating with the HostServer.
7. Clicking on the associated link from the NameServer puts a user in direct contact with the listed HostServer or Agent, thereby re-establishing contact with the agent.
8. If you do not implement zombies, then when an agent migrates, the user will submit to a dead link, and will have to re-establish contact with an agent through the NameServer.
9. If you DO use zombies, (see Zombie Example ) then when an agent migrates, on reconnection, the user will get an intermediate page, possibly with all of their data intact, which will now point to the new location of the agent.

Ports

• IP address: localhost (127.0.0.1) for grading, by default. Optional: other at home. Pass as second argument to new HostServers
• User communication Port: 48050
• Service Port: 48060

• Default Port: 45050 or by passing as first argment (will use 45050-45060)
• IP address: localhost (127.0.0.1) for grading -- but modify at home as needed

• User communication port: determined by current HostServer at hosting time, but see Get next available port
• Agent communication port: determined by current HostServer at hosting time
• Optional administration port: determined by current HostServer at hosting time

Sharing data

• Is the data stored on each agent, with updates broadcast to all?
• Is the data stored in one location with agents making retrievals over the network for each display request? If so, how is this managed? (If you need a separate server to manage shared data, call it DIAdb.java.)
• Do you broadcast new data to all the agents every time there is an update at one of the agents?
• If you implement a model of agent-death, what happens to the group's data when the last agent in a group goes away?

The lunch problem

1. A user should submit the label for the data before entering the matching data itself. This will help avoid the problem that the user enters a label AND the data, submits the update, and then we discover that there is already a description for that label.

2. However, because data is shared between agents in a group, we can have the following problem: User Alice submits the label "UDP" and, after a check is made to verify that the label "UDP" does not already exist in the database (DB), she is prompted to enter the matching data (description) for that concept. Meanwhile User Bob submits a request to update the DB for "UDP" and is given his own update screen for that concept. User Alice submits her matching data, which is indexed under "UDP". User Bob submits his matching data for "UDP" which then writes on top of Alice's submission. Alice goes back to read her data, sees Bob's matching data, and gets upset. Where is her data that she just typed in?

3. So we decide to use record locking. User Alice submits the label "UDP" and she is prompted to enter the matching data (description) for that concept. Meanwhile User Bob submits a request to update the DB for "UDP" and is told that the database entry for "UDP" has been locked. Meanwhile Alice decides to go out to lunch. Then, when she comes back she gets involved in something else, then goes home, and leaves on vacation the next day. Bob repeatedly tries to either retrieve the data for "UDP" or enter his own, but is locked out for two weeks while Alice is on vacation. During this time, Carol and Dave discover that they are locked out as well. This is the classic lunch problem .

4. In the case of this particular application, having a single record locked, or losing a small amount of data, is not fatal to the application. But the problem occurs in many kinds of update environments, and should be handled both intentionally, and with grace, on your part in this pedagogical exercise.

5. A number of devices might be used in your solution. (a) Is there a time-out on the lock? Might you wish to lock the whole database? Are you willing to take on the burden of integrating disparate data when two users submit in an overlapping way?

6. Where are the bottlenecks? How will your solution scale to, say, 100,000 agents, and 50,000 concepts? Where are the weakpoints of your system? Can the failure of one node cause the whole system to go down?

HostServers

Migration

Keep in mind that when an agent migrates from one location to another (e.g., to a different HostServer) the code that implements the agent is already there. Migration, in our case here, means shipping the state of the agent (ID number, group, relationships, data, history, etc.) to be instantiated into the agent code that is started up at the new location.

1. The easiest way to do this, although not elegant, is to simply leave a dead link. The user must then be educated enough to know that they have to connect to the NameServer to find the current location of their agent. Using this scheme, no notification that the agent has migrated is possible.
2. A somewhat more elegant method is to use an automatic redirection scheme that sends back dynamically-generated html that points to the new (basic) endpoint of the agent. However, if we want to keep things simple, this will only be a top-level communication with the agent (that is, no action statement, no form data), and the user's currently submitted data may be lost.
3. A more elegant method is to leave behind a zombie thread, listening at the old port, which redirects the browser to the new endpoint of the agent. (see Zombie Example. )
   • A simple scheme, and very acceptable for our purposes here, is to send back, from the zombie, dynamically reconstructed html that includes all of the user's just-submitted attribute/value pairs (data), but replaces the action-statement URL with that of the new location of the agent, and ask the user to resubmit it. A simple banner saying something like "Your agent has migrated, press the submit button to reconnect" with a collection of data using hidden or displayed values would be fine. This way your user will not lose data, or the state of their conversation with the agent.
   • If you maintain an admin connection to your agent, the zombie will need to wait for this connection too.
   • The zombie does not need to wait for agent communications because agents can be notified of the new location before the migration.
   • Lastly, if you really wanted to make this elegant, you could combine techniques: have the zombie parse the type of request from the browser and forward all of the data submitted to the newly-migrated agent; send a redirect screen to the browser to connect to the agent's new location; return an acknowledgment of the data submission from the agent to the browser.
4. You will have to think through the sequence of events that have to take place for migration to occur. Consider that some sequences will be more problematic if there are lost communications, or a process fails in the middle of the migration. In your comments, discuss your system's robustness, or lack thereof, in the presence of different types of failures during migration. (For example, the NameServer provides a new destination host location, and also updates its tables. Does this all take place in one request, or is the old table information maintained until after the migration is complete, requiring at least two connections to the NameServer?)
5. Implement automous migration by using a Sleep Looper . The agent wakes up, and may or may not decide to migrate.

Server Logs

User Communication

1. Cold connection to the agent: dump the agent's state (except for data): group affiliation, other agents in the group, number of user requests for getting data, and also for posting data, history of migrations, number (or log) of autonomous communications from other agents. Provide links to move into to get-data mode, or send-data mode.
2. Get label for data (e.g. pose a quiz question). Labels should repeat until all have been seen by the user.
3. Get matching data (e.g., present the associated answser - include the label as well).
4. Check a label for redundancy
5. Submit matching data (after checking the label - include the label at the top, and a data-input box below it).

• Top-level Connection
• Display Label
• Display Matching Data
• Check Label for redundancy
• Enter Data

1. Your agent will be getting different types of requests through the same port, at least: (a) requests for a new label to be displayed (b) requests for matching data to be displayed (c) requests for a label to be checked for redundancy before entering matching data (d) requests to post a label/content pair (send data), and (e) requests to reconnect to an agent (i.e., from NameServer link referral ).
2. Imagine that a client connects to request a label, and your agent sends back a page saying "UDP". The user thinks about this label, forms her opinion and then presses submit to get the "answer" to see if she was right. How do you know to send back the matching data that says, e.g., "Cheap sending of datagrams without guarantee of arrival or ordering." ? There are many ways of handling this problem. For example you might...
   1. Send the matching data along with the original label, but put it in a hidden value. Then when the user submits the page asking for the matching data to be displayed, they are actually sending the data they want in the request This way, even if you have migrated, or the agent was restarted, you always have the data at hand to be simply returned to them.
   2. Do an expensive string match on the original label (which is resent by the user), and send back the matching data when you locate it. This way even if you have migrated, or the agent has been restarted, you still know exactly what to send back.
   3. Generate a random number and store a permanent copy of it as an ID in the structure where you store your copy of label and the data. When the user gets the label, they also get the ID number. Then, when the user presses submit they send not the label, but the ID number. You then make this cheap (possibly indexed) match against the ID number, retrieve the associated matching data and send it back to the user.
   4. Simply wait for the user to send another request and when they do, as long as it is for matching data, just send back the matching data that you have already queued. This simple mechanism will work, as long as the client browser and your agent stay in sync.
3. (Note: Some of you may optionally implement an administration client for agents. All of the migration issues for the user client also apply to the aministration client.)
4. This design requires the sending of a great deal dynamically-generated HTML from the agents back to the users. Are you clear on how this is done? (The essence of it: use print-stream functions to send primarily static HTML strings from your program. Fill in a few slots with dynamically-generated strings.)

Communication Data: