Click here to load JAG Reloaded
Who's Online
10 registered (Dyson, kapratko, Gorro der Grüne, K0ukku, Mauser, 2 invisible), 20 Guests and 6 Spiders online.
Key: Admin, Global Mod, Mod
Page 1 of 3 1 2 3 >
Topic Options
Rate This Topic
#181448 - 15 April, 2008 03:31 AM prof.dat -> xml
BirdFlu
Lieutenant
Lieutenant

Posts: 598
Loc: Lampukistan
Hi,

after reading some threads on this board, i thought there might be a need for a non-binary prof.dat file. I converted the original file
to a xml file, but the results are a little unsatisfactory. First, the file is huge (170 merc entries with over 70000! lines in total).
And second, the order of entries is not really intuitive, but that is a result of the order in the "OLD_MERCPROFILESTRUCT_101" class.

Here is a small example:
 Code:
<?xml version="1.0" encoding="utf-8" ?>
<root>
  <MERCPROFILE index="0" nickname="Barry" version="101">
    <zName> Barry Unger </zName>
    <zNickname> Barry </zNickname>
    <uiAttnSound> 17 </uiAttnSound>
    <uiCurseSound> 5308432 </uiCurseSound>
    <uiDieSound> 20 </uiDieSound>
    <uiGoodSound> 852000 </uiGoodSound>
    <uiGruntSound> 852030 </uiGruntSound>
    <uiGrunt2Sound> 7012600 </uiGrunt2Sound>
    <uiOkSound> 17 </uiOkSound>
    <ubFaceIndex> 98 </ubFaceIndex>
    <_PaletteRepID>
      <PANTS> TANPANTS </PANTS>
      <VEST> BLUEVEST </VEST>
      <SKIN> PINKSKIN </SKIN>
      <HAIR> BLACKHEAD </HAIR>
    </_PaletteRepID>
    <bSex> 0 </bSex>
    <bArmourAttractiveness> 100 </bArmourAttractiveness>
    <ubMiscFlags2> 0 </ubMiscFlags2>
    <bEvolution> 0 </bEvolution>
    <ubMiscFlags> 0 </ubMiscFlags>
    <bSexist> 3 </bSexist>
    <bLearnToHate> -1 </bLearnToHate>
    <_skills>
      <bStealRate> 0 </bStealRate>
      <bVocalVolume> 0 </bVocalVolume>
      <ubQuoteRecord> 0 </ubQuoteRecord>
      <bDeathRate> 22 </bDeathRate>
      <bScientific> 0 </bScientific>
    </_skills>
    <_gain>
      <sExpLevelGain> 0 </sExpLevelGain>
      <sLifeGain> 0 </sLifeGain>
      <sAgilityGain> 0 </sAgilityGain>
      <sDexterityGain> 0 </sDexterityGain>
      <sWisdomGain> 0 </sWisdomGain>
      <sMarksmanshipGain> 0 </sMarksmanshipGain>
      <sMedicalGain> 0 </sMedicalGain>
      <sMechanicGain> 0 </sMechanicGain>
      <sExplosivesGain> 0 </sExplosivesGain>
    </_gain>
    <ubBodyType> 0 </ubBodyType>
    <bMedical> 20 </bMedical>
    <usEyesX> 10 </usEyesX>
    <usEyesY> 8 </usEyesY>
    <usMouthX> 7 </usMouthX>
    <usMouthY> 28 </usMouthY>
    <uiEyeDelay> 0 </uiEyeDelay>
    <uiMouthDelay> 0 </uiMouthDelay>
    <uiBlinkFrequency> 3000 </uiBlinkFrequency>
    <uiExpressionFrequency> 2000 </uiExpressionFrequency>


Before i continue my work, i thought i should ask how other want it to have (if you want it at all).
Offline - OFFLINE
Top
Badges:
#181449 - 15 April, 2008 03:35 AM Re: prof.dat -> xml [Re: BirdFlu]
Kaerius
Sargeant
Sargeant

Posts: 199
I'd like it in XML format yeah... and integrated into the XML editor as well hopefully, otherwise what's the point? That pretty much sums it up with requests from me, as anything extra I'd want can be handled by the editor(like say if I wanted the mercs listed alphabetically, it's available at the press of a button in the editor).
Offline - OFFLINE
Top
#181451 - 15 April, 2008 04:06 AM Re: prof.dat -> xml [Re: Kaerius]
Mauser
Major
Major

Posts: 2233
Loc: Bavaria - Germany
well, modmakers are waiting for an externalized, fully moddable prof.dat for a long time now.

itīs one of the essential elements left for externalization required for serious modding.

so itīs a real blessing, if this would be finally made available now. but yes, we need a compatible editor with that if possible.

and if that isnīt possible to do without major complications, then another very important thing to do would be the externalization of the laptop files, websites, emails and reports and such.

this may actually be more easy to do and more easy to convert into XML files.

anyways, thanks a lot BirdFlu and keep up the good work!
_________________________
though I walk thru the valley of the shadow of death, I shall fear no evil, 'cause I'm the meanest s.o.b. in the valley.

Semi official Ambassador, Consultant (and errand boy) of the SMP Project

Online   - ONLINE
Top
Badges:
#181462 - 15 April, 2008 04:48 AM Re: prof.dat -> xml [Re: Mauser]
Khor1255
General
General

Posts: 5954
Loc: Pleasantville, NJ
I don't agree that we need an editor especially right away. In many ways I liked the xmls before the editor came along.

What we need is clear headers describing each field of the prof.xml and NEVER DO AWAY WITH THESE HEADERS.

Once we have this, we really have all we need.

Sure, sometime down the road a gui might be cute as well but please PLEASE do not break the xml to make the gui.

THe headers are very important and to me being able to edit the .xml in a simple program like notepad is golden. I don't want to wait 20 seconds for some fancy program to boot up just to make a feww changes I catch during gameplay testing. With the items.xmls it was so much easier for me just to open them in notepad make the changes and get on with more modding.

We don't need a gui we need an externalised prof.dat.

If you are doing this you are the shit!
_________________________
Dan Watson
Offline - OFFLINE
Top
#181473 - 15 April, 2008 05:15 AM Re: prof.dat -> xml [Re: Khor1255]
Lt.Havoc
Sargeant
Sargeant

Posts: 145
Loc: Germany
Nope, I say we need a GUI, then I dont know how to edit XML in notepad! We are not all Profi coders and modders who mod games efore thier first coffee, ya know. Also, what are 20 sec of wating? Notepad needs the same amount of time to load a huge XML file. The XML editor is one of the best things I ever saw, then its really easy to use that every dumbass can change the XML files.

So, if we externalized the prof dat, I also want XML editor support so I can change everything I want without scrolling though endless lines of numbers and letters, I mean, you can eaisly search with the editor, not with a notepad.

So, I say, brake it so it fits a GUI.
_________________________
"Go ahead, make my day!" Harry Callahan

"This is a .44 Magnum, the most powerful handgun in the world. It can blow your head clean off. Do you feel lucky, punk?" Harry Callahan
Offline - OFFLINE
Top
#181479 - 15 April, 2008 05:28 AM Re: prof.dat -> xml [Re: Lt.Havoc]
Khor1255
General
General

Posts: 5954
Loc: Pleasantville, NJ
You don't have to know how to do anything other than read English to edit an .xml file in notepad.

If making a gui slows this project down even two hours I say that is two hours wasted time.

For most casual editing all you need is proedit anyway so please explain to me what you think you will be using the prof.xml for anyway?
_________________________
Dan Watson
Offline - OFFLINE
Top
#181483 - 15 April, 2008 05:32 AM Re: prof.dat -> xml [Re: Khor1255]
Lt.Havoc
Sargeant
Sargeant

Posts: 145
Loc: Germany
You cant use proedit with 1.13 dat files and NIV, because it dosent regonize the files. The 1.13 files are diffrent compared to the orginal ones and as long as no one redoes the orginal proedit to work with 1.13, You cant edit your IMPs to be buddies etc.

This mod is for modmakers and should be easy to accsess, so the easiest way would be to do that is to have an full blown editor.
_________________________
"Go ahead, make my day!" Harry Callahan

"This is a .44 Magnum, the most powerful handgun in the world. It can blow your head clean off. Do you feel lucky, punk?" Harry Callahan
Offline - OFFLINE
Top
#181486 - 15 April, 2008 05:38 AM Re: prof.dat -> xml [Re: Lt.Havoc]
Khor1255
General
General

Posts: 5954
Loc: Pleasantville, NJ
I don't know about niv but you can definately use proedit with the 1.13.

You just have to name the prof.dat file to the difficulty and guns setting you use. It is very simple. Please refer to the times it has been explained here or have someone explain it.

Or


Open proedit
make your changes
save
copy/paste your prof.dat file (may just be called prof if you have file extensions turned off in your computer)
rename the file to (for example) ProfExpeiencedTonsOfGuns.dat if that is the way you play (on experienced level with tons of guns)
enjoy

I con't believe they broke that in the niv but if they did it is a shame. I know for a fact it works in regular 1.13 and always has.
_________________________
Dan Watson
Offline - OFFLINE
Top
#181488 - 15 April, 2008 05:39 AM Re: prof.dat -> xml [Re: Khor1255]
Mauser
Major
Major

Posts: 2233
Loc: Bavaria - Germany
 Originally Posted By: Khor1255
If making a gui slows this project down even two hours I say that is two hours wasted time.


well, in that respect i somehow agree with you completely Khor1255.

we need this externalizations really badly and having them asap without an easy to use editor is still better than having them a lot later at all.

so, the experienced modders and XML hackers can already work with it which definitely will benefit the mods currently under developement.

and the less capable modders will just have to wait for the comfort version then.
_________________________
though I walk thru the valley of the shadow of death, I shall fear no evil, 'cause I'm the meanest s.o.b. in the valley.

Semi official Ambassador, Consultant (and errand boy) of the SMP Project

Online   - ONLINE
Top
Badges:
#181490 - 15 April, 2008 05:47 AM Re: prof.dat -> xml [Re: Mauser]
Khor1255
General
General

Posts: 5954
Loc: Pleasantville, NJ
As long as there are clearly written headers explaining each field of the xml it really does not require any special knowledge at all. it was when they first removed the headers then made the xml entry lines all run on without any table like listing that things got almost impossible for me.

The xml editor is still clunky compared to the way I used to do it but for items I can at least see why players would want this.

But really, the proedit program does just about everything you would want as a player so I do not see where the casual user would even benefit from a prof.xml.

But those of us working on large scale mods really need this and the sooner the better.
_________________________
Dan Watson
Offline - OFFLINE
Top
#181493 - 15 April, 2008 06:09 AM Re: prof.dat -> xml [Re: Khor1255]
BirdFlu
Lieutenant
Lieutenant

Posts: 598
Loc: Lampukistan
Easy guys. The xml editor is the last thing we need (i said the LAST, i didn't say we DON'T need it). There is still a lot that has to be done before.

We have to define a proper structure for the files. I don't really like long parameter lists, that is if you consider the xml file as a tree
(not a real tree), then i would rather have depth (not too deep) instead of breadth.

Once the file structure is stable (only the basic structure, the order of the children of a node should not matter much), we actually have
to USE the file in the game. Without that the whole discussion is pointless. We also have to decide where we want to use the files.
The data in the prof.dat file describe only the initial stats of a merc, but these stats change after some while. These changes are
saved in the savegames, i suppose. So maybe we want to save the changed merc profiles as xml files too, or maybe even save the
savegames itself as xml files.

When all this is done, then we can start with the xml editor. Until then just grab a regular xml editor, it will do the job just fine.
Offline - OFFLINE
Top
Badges:
#181534 - 15 April, 2008 08:39 AM Re: prof.dat -> xml [Re: BirdFlu]
the scorpion
General
General

Posts: 5492
Loc: CH

______
BANNED
i'm not really sure about prof.dat anymore.

these days, i personally would prefer externalising things about characters that ARE NOT EVEN IN PROF.DAT YET but hardcoded in the actual exe first, otherwsie, prof.dat in xml format will still bear a lot of severe restrictions

maybe that shoud come first, felxibility first, then the possibility to amend stuff to prof.dat

take for example the playable character slots. If they were arbitrary rather than limited to very specific numbers, modmakers would gain countless times more than what they would if 10 characters can be amended to the list in prof.dat

take for example the hardcoded npc/ rpc placement routines: if they would be editable, modmakers could use all existing slots for all purposes

the same about trader ID's. make any unused char a trader as we like in, say, traders.xml.

all of these things are in fact more limiting that prof.dat/ proedit the way it is now.



maybe both can be done at once: when you externalisze prof.dat to new xml's, maybe externalising those important values that are not even in prof.dat as well to the same xml's would be the way to go. But it would complicate things even more.

as mentioned before, i'm unsure as to what has to be done first. what's important is to find a solution that gives the most gain. There are many ways in which prof.dat hinders other development, that's why it should be bought into an easier format.


people have accused me that viewing things that way is shortsighted. But i'd like to have brought the aspects enumerated above to people's knowledge before prof.dat is externalised and people recognise that there are still various limitations.
Offline - OFFLINE
Top
#181569 - 15 April, 2008 11:27 AM Re: prof.dat -> xml [Re: the scorpion]
BirdFlu
Lieutenant
Lieutenant

Posts: 598
Loc: Lampukistan
@ the scorpion : are you talking about having an arbitrary number of mercs with arbitrary/multiple group affiliations?

<for_coders>
The two main data structures in the code are the "NPCIDs" (enum) and "gMercProfiles" (MERCPROFILESTRUCT[NUM_PROFILES]).
The merc profile array is used all over the code, thus changes should be as non-invasive as possible. But i think this
array can be replaced with a class (object) that overloads the [] operator :
 Code:
 
	MERCPROFILESTRUCT& MercProfileClass::operator[](int ID);
	MERCPROFILESTRUCT& MercProfileClass::operator[](string Name, bool bFullName=false); // name or nickname

The effect on the outer world should be minimal. And inside the class we can do whatever we want, like
choosing a profile order that does not depend on any predefined data. We only need to handle the "NPCIDs"
properly, so maybe we could just register them in our new class in order to identify them later.
</for_coders>

Now we have the following data in prof.dat (multiplied 170 times)
 Code:
	CHAR16		zName[ NAME_LENGTH ];
	CHAR16		zNickname[ NICKNAME_LENGTH ];
	UINT32		uiAttnSound;
	UINT32		uiCurseSound;
	UINT32		uiDieSound;
	UINT32		uiGoodSound;
	UINT32		uiGruntSound;
	UINT32		uiGrunt2Sound;
	UINT32		uiOkSound;
	UINT8		ubFaceIndex;
	PaletteRepID	PANTS;
	PaletteRepID	VEST;
	PaletteRepID	SKIN;
	PaletteRepID	HAIR;
	INT8		bSex;
	INT8		bArmourAttractiveness;
	UINT8		ubMiscFlags2;
	INT8		bEvolution;
	UINT8		ubMiscFlags;
	UINT8		bSexist;
	INT8		bLearnToHate;

	// skills
	INT8		bStealRate;
	INT8		bVocalVolume;
	UINT8		ubQuoteRecord;
	INT8		bDeathRate;
	INT8		bScientific;

	INT16		sExpLevelGain;
	INT16		sLifeGain;
	INT16		sAgilityGain;
	INT16		sDexterityGain;
	INT16		sWisdomGain;
	INT16		sMarksmanshipGain;
	INT16		sMedicalGain;
	INT16		sMechanicGain;
	INT16		sExplosivesGain;

	UINT8		ubBodyType;
	INT8		bMedical;

	UINT16		usEyesX;
	UINT16		usEyesY;
	UINT16		usMouthX;
	UINT16		usMouthY;
	UINT32		uiEyeDelay;
	UINT32		uiMouthDelay;
	UINT32		uiBlinkFrequency;
	UINT32		uiExpressionFrequency;
	UINT16		sSectorX;
	UINT16		sSectorY;

	UINT32		uiDayBecomesAvailable;	//day the merc will be available.	used with the bMercStatus

	INT8		bStrength;

	INT8		bLifeMax;
	INT8		bExpLevelDelta;
	INT8		bLifeDelta;
	INT8		bAgilityDelta;
	INT8		bDexterityDelta;
	INT8		bWisdomDelta;
	INT8		bMarksmanshipDelta;
	INT8		bMedicalDelta;
	INT8		bMechanicDelta;
	INT8		bExplosivesDelta;
	INT8		bStrengthDelta;
	INT8		bLeadershipDelta;
	UINT16		usKills;
	UINT16		usAssists;
	UINT16		usShotsFired;
	UINT16		usShotsHit;
	UINT16		usBattlesFought;
	UINT16		usTimesWounded;
	UINT16		usTotalDaysServed;

	INT16		sLeadershipGain;
	INT16		sStrengthGain;



	// BODY TYPE SUBSITUTIONS
	UINT32		uiBodyTypeSubFlags;
	
	INT16		sSalary;
	INT8		bLife;
	INT8		bDexterity;		// dexterity (hand coord) value
	INT8		bPersonalityTrait;
	INT8		bSkillTrait;

	INT8		bReputationTolerance;
	INT8		bExplosive;
	INT8		bSkillTrait2;
	INT8		bLeadership;

	INT8		bBuddy[5];
	INT8		bHated[5];
	INT8		bExpLevel;		// general experience level

	INT8		bMarksmanship;
	UINT8		bMinService;
	INT8		bWisdom;
	UINT8		bResigned;
	UINT8		bActive;

	UINT8		DO_NOT_USE_bInvStatus[OldInventory::NUM_INV_SLOTS];
	UINT8		DO_NOT_USE_bInvNumber[OldInventory::NUM_INV_SLOTS];
	UINT16 		usApproachFactor[4];

	INT8		bMainGunAttractiveness;
	INT8		bAgility;		// agility (speed) value

	BOOLEAN		fUseProfileInsertionInfo;	// Set to various flags, ( contained in TacticalSave.h )
	INT16		sGridNo;			// The Gridno the NPC was in before leaving the sector
	UINT8		ubQuoteActionID;
	INT8		bMechanical;

	UINT8		ubInvUndroppable;
	UINT8		ubRoomRangeStart[2];
	UINT16 		DO_NOT_USE_inv[OldInventory::NUM_INV_SLOTS];
	INT8 		bMercTownReputation[ 20 ];

	UINT16 		usStatChangeChances[ 12 ];	// used strictly for balancing, never shown!
	UINT16 		usStatChangeSuccesses[ 12 ];	// used strictly for balancing, never shown!

	UINT8		ubStrategicInsertionCode;

	UINT8		ubRoomRangeEnd[2];

	INT8 		bPadding[ 4 ];

	UINT8 		ubLastQuoteSaid;
	
	INT8 		bRace;
	INT8 		bNationality;
	INT8 		bAppearance;
	INT8 		bAppearanceCareLevel;
	INT8 		bRefinement;
	INT8 		bRefinementCareLevel;
	INT8 		bHatedNationality;
	INT8 		bHatedNationalityCareLevel;
	INT8 		bRacist;
	UINT32 		uiWeeklySalary;
	UINT32 		uiBiWeeklySalary;
	INT8 		bMedicalDeposit;
	INT8 		bAttitude;
	INT8 		bBaseMorale;
	UINT16 		sMedicalDepositAmount;
	
	INT8 		bLearnToLike;
	UINT8 		ubApproachVal[4];
	UINT8 		ubApproachMod[3][4];
	INT8 		bTown;
	INT8 		bTownAttachment;
	UINT16 		usOptionalGearCost;
	INT8 		bMercOpinion[75];
	INT8 		bApproached;
	INT8 		bMercStatus;		// The status of the merc. 
						// If negative, see flags at the top of this file.	
						//   Positive :	The number of days the merc is away for.	
						//   0        :	Not hired but ready to be.
	INT8 		bHatedTime[5];
	INT8 		bLearnToLikeTime;
	INT8 		bLearnToHateTime;
	INT8 		bHatedCount[5];
	INT8 		bLearnToLikeCount;
	INT8 		bLearnToHateCount;
	UINT8 		ubLastDateSpokenTo;
	UINT8 		bLastQuoteSaidWasSpecial;
	INT8		bSectorZ;
	UINT16 		usStrategicInsertionData;
	INT8 		bFriendlyOrDirectDefaultResponseUsedRecently;
	INT8 		bRecruitDefaultResponseUsedRecently;
	INT8 		bThreatenDefaultResponseUsedRecently;
	INT8 		bNPCData;			// NPC specific
	INT32		iBalance;
	INT16 		sTrueSalary; // for use when the person is working for us for free but has a positive salary value
	UINT8		ubCivilianGroup;
	UINT8		ubNeedForSleep;
	UINT32		uiMoney;
	INT8		bNPCData2;			// NPC specific

	UINT8		ubMiscFlags3;

	UINT8 		ubDaysOfMoraleHangover;		// used only when merc leaves team while having poor morale
	UINT8		ubNumTimesDrugUseInLifetime;	// The # times a drug has been used in the player's lifetime...

	// Flags used for the precedent to repeating oneself in Contract negotiations.	
	// Used for quote 80 -	~107.	Gets reset every day
	UINT32		uiPrecedentQuoteSaid;
	UINT32		uiProfileChecksum;
	INT16		sPreCombatGridNo;
	UINT8		ubTimeTillNextHatedComplaint;
	UINT8		ubSuspiciousDeath;

	INT32		iMercMercContractLength;	//Used for MERC mercs, specifies how many days the merc has gone since last page

	UINT32		uiTotalCostToDate;		// The total amount of money that has been paid to the merc for their salary
	UINT8		ubBuffer[4];

Writing this values as they are into a xml file would result in a file with similar structure (long list of parameters). But i would rather have something like this
 Code:
MERCPROFILESTRUCT:
  Name
  NickName
  MercTraits:
    Attributes:
      Strength
      Wisdom
      ...
    Skills
      Mechanical
      Explosives 
      ...
    ...
  GameData
    Sounds 
      OkSound
      DieSound
      ...
    FaceOffsets
      EyesX
      EyesY
      MouthX
      ...
    ...
  ...

One way to structure the variables is to group the values as it is done in proedit.exe, but maybe we can come up with something better.
And of course we are not restricted by prof.dat and can include any other data that can be linked to a merc (in a reasonable way).
Offline - OFFLINE
Top
Badges:
#181571 - 15 April, 2008 11:58 AM Re: prof.dat -> xml [Re: BirdFlu]
the scorpion
General
General

Posts: 5492
Loc: CH

______
BANNED
 Originally Posted By: BirdFlu
@ the scorpion : are you talking about having an arbitrary number of mercs with arbitrary/multiple group affiliations?


don't know what you mean by "group affiliation"

what i was suggesting is to get rid of the very severe slot assignements all characters in prof.dat have. I guess this has to do with the NPCID enums.
i'd like to be able to use any character slot for any type of character with any type of placement routine. ATM, all of these important factors are hardcoded and cannot be changed in proedit.

if they could, we'd gain a lot of flexibility in the handling of the slots even before prof.dat has to be touched.

of course these restrictions go clearly beyond prof.dat, and i was thinking aloud whether it is better to have these restrictiosn cleaned out first before prof.dat is restructured.
Offline - OFFLINE
Top
#181576 - 15 April, 2008 12:11 PM Re: prof.dat -> xml [Re: the scorpion]
Lesh 2
N.C.O
N.C.O.

Posts: 311
Loc: Izhevsk, Russia
 Originally Posted By: the scorpion
maybe both can be done at once: when you externalisze prof.dat to new xml's, maybe externalising those important values that are not even in prof.dat as well to the same xml's would be the way to go. But it would complicate things even more.

as mentioned before, i'm unsure as to what has to be done first. what's important is to find a solution that gives the most gain. There are many ways in which prof.dat hinders other development, that's why it should be bought into an easier format.

people have accused me that viewing things that way is shortsighted. But i'd like to have brought the aspects enumerated above to people's knowledge before prof.dat is externalised and people recognise that there are still various limitations.

I understand you, Scorpion.
I think, the first to be done is an overview of an game aspects, somehow related to prof.dat ( = character list). Let me start.

Ja2 has many types of game characters:
- AIM mercs;
- MERC mercs;
- RPC's;
- robot;
- vehicle;
- player chars;
- NPC's (RPC's), that has subcategories:
--- traders, repairers;
--- terrorists;
--- assasins;
--- miners;
--- ordinary npc, that is involved into some quest;
--- special npc, that is shown in meanwhile scene;
... maybe forgot something ...

My point is that prof.dat is a middle-ware, a character list. This list holds various attributes and stats, used by every char in game. But it lacks something. Prof.dat doesn't contain information about role of the character (and it's associated data) in the game. So there must be high-level structures, like miner's list, that refers to character list and points, which characters are miners.

But I said, prof.dat is a middle-ware. There is low-level structures, referred by prof.dat itself, a faces list. The faces list contain information about eyes and mouth (x,y) coordinates for big and small faces, face expression.

The current problem with faces is that some characters have only small faces (mercenaries), some characters have only big faces (non-recruitable npc's), some characters have both big and small faces (recruitable npc's), some characters doesn't have animated face (robot, vehicle) and the last, special character Elliot has several (!!!) faces. His face is changing during the campaign, as the queen slaps him more and more.

So final scheme:

role --> character --> face

Let's discuss!
_________________________
JA2 v1.13 for linux
Bugreports (zip, gz, bz2): ja2113bugreports@googlemail.com. In letter: description and other (usually, saved games and logs)
Offline - OFFLINE
Top
#181583 - 15 April, 2008 12:41 PM Re: prof.dat -> xml [Re: Lesh 2]
ChrisL
Captain
Captain

Posts: 1957
@BirdFlu: Couple things. First, as of the NIV merge, starting equipment has been moved from prof.dat over to MercStartingGear.xml at least while in NIV mode. It would be simple enough to update the code so that MercStartingGear.xml was used regardless of the inventory mode. I simply left it as an NIV only system so that people who prefered OIV and the existing prof.dat could use them.

Prof.dat (and mercstartinggear.xml) only equate to profile data. These files are only read once when a new game is started. From that point on, all profile information is stored in the gMercProfiles array. This array is part of every save.

Lastly, there's one other thing you should consider. At the present, we actually use 8 different prof.dat files. There's a different prof.dat for each difficulty and equipment mode. Assuming you wanted to keep that same functionality, you'd be looking at 8 different files, each with 70k+ lines. This is one of the main reasons I didn't try to completely externalize prof.dat when I setup the mercstartinggear.xml file. More then converting the existing prof.dat files over to xml, I'd rather see someone create a new prof.dat editor that pulled data from the existing xml files, and included functionality to read all 8 different versions of the prof.dat files.
Offline - OFFLINE
Top
Badges:
#181610 - 15 April, 2008 03:03 PM Re: prof.dat -> xml [Re: ChrisL]
BirdFlu
Lieutenant
Lieutenant

Posts: 598
Loc: Lampukistan
 Originally Posted By: the scorpion
don't know what you mean by "group affiliation"

being a member of a group, like rebels, traders, police, army, AIM, MERC, barkeeper guild etc.

 Quote:
.. MercStartingGear.xml ..

I thought about splitting the file too, but where should the limit be, 2 files or maybe 5 or 10. Or we create a file for each merc (-> 170 files).

 Quote:
Prof.dat (and mercstartinggear.xml) only equate to profile data. These files are only read once when a new game is started. From that point on, all profile information is stored in the gMercProfiles array. This array is part of every save.

Is that good or bad? Do we want to change mercs profiles after game start? Should savegames be binary or xml files (compressed, to keep them small)?

 Quote:
.. 8 prof.dat files ..

we could include everything into one file, so we would not need to duplicate data that doesn't change.
 Code:
  ...
  <marksmanship>
    <value difficulty="Novice"> 80 </value>
    <value difficulty="Experienced"> 75 </value>
    <value difficulty="Expert"> 70 </value>
    <value difficulty="Insane"> 50 </value>
  </marksmanship>
  ...
  <startinggear>
    <vest>
      <value difficulty="Novice" guns="TonsOfGuns"> 20 </value>
      <value difficulty="Experienced" guns="any"> 30 </value>
      ...
    </vest>
  </startinggear>

Of course the parsing would take longer, but we have to decide at a point, many smaller files or one big file, or something in between of course.

 Quote:
More then converting the existing prof.dat files over to xml, I'd rather see someone create a new prof.dat editor that pulled data from the existing xml files, and included functionality to read all 8 different versions of the prof.dat files.

But that doesn't affect the internal processing of the files inside the game.
Offline - OFFLINE
Top
Badges:
#181622 - 15 April, 2008 03:35 PM Re: prof.dat -> xml [Re: BirdFlu]
the scorpion
General
General

Posts: 5492
Loc: CH

______
BANNED
these "groups" you mention are very different. some of them are just factions as defined in prof.dat, others are hardcoded slot limitations.



i'd really like you guys to keep in mind what lesh wrote up there. There is a lot of important data at a "higher" level than prof.dat that needs to be adressed.

and i was wondering whether it'd be necessary to externalise this first.


lesh mentioned the most important case, the character's "role", or, better, what one can do with a character on a specific slot.
I think this is extremely important and steps forward in this department could help a lot when adressing prof.dat

then something else is placement routines. If you look at prof.dat, you typcially encounter two different possible entries for the sector in which the character appears: an explicit sector on the one hand, and "@0" on the other hand.

Now every @0 character has a hardcoded placement routines which can only very unreliably and in very crude ways be changed.

This is something that is typcially not really editable in prof.dat, even if you set a specific sector, you still get the hardoded placement routine

this effecttivly keeps many slots from being used arbitrarily and restricts many a slot.


then what lesh also mentions is face coords. Only AIM and MERC characters have their face coords defined by prof.dat, RPC's, IMP's and EPC's have theirs hardcoded.
if we want to be able to expand on prof.dat, which seems the main point in porting it to .xml, there must be ways to define new coords for new RPC chars. In order to define new RPC chars, we must be somewhat flexible with the slots...

all of these things have very rigid ties to each other. I think it may have to be deal with as a whole package in order to achieve the best results.
Offline - OFFLINE
Top
#181623 - 15 April, 2008 03:38 PM Re: prof.dat -> xml [Re: BirdFlu]
Khor1255
General
General

Posts: 5954
Loc: Pleasantville, NJ
@Bird Flu

You are definately on the right track.

Factions was the term you were looking for when you said affiliation.

scorpion brings up very important points here about things that may not be contained in the prof.dat. But I think this could be 'easily' remedied by child directories such as we see springing from the items.xml.

If you could perhaps make a quests.xml, events.xml, mapcharacter.xml (stupid name but I think you get that I mean how the character is interdependant/reacts to map placement and other maps (i.e. Joey returns to Martha's sector to complete his quest etc.), npc.xml (this should be a doozie since it would cover what the .npc files currently do, etc.

Having all these 'spring' from the Prof.xml 'trunk' would be a way to illustarate/externalise them into an editable format. Right?
_________________________
Dan Watson
Offline - OFFLINE
Top
#181631 - 15 April, 2008 04:30 PM Re: prof.dat -> xml [Re: Khor1255]
BirdFlu
Lieutenant
Lieutenant

Posts: 598
Loc: Lampukistan
 Quote:
Factions was the term you were looking for when you said affiliation.

I actually meant more than just factions. Inside a faction there could be more subgroups like admins, elites etc.,
and a merc can be a member of multiple groups (nationalities, employer, etc.)

 Quote:
then something else is placement routines. If you look at prof.dat, you typcially encounter two different possible entries for the sector in which the character appears: an explicit sector on the one hand, and "@0" on the other hand.

Now every @0 character has a hardcoded placement routines which can only very unreliably and in very crude ways be changed.

This is a good point. @0 actually means (sSectorX=0, sSectorY=0) in the file and is an invalid sector, i suppose. That means that
the real numbers have to be determined in the code and the game just does it. I found the "HandleEarlyMorningEvents" function that sets
sector values for some mercs by using their hardcoded ID. Now, if we would refer to them by name (instead of their ID), then this
problem would go away just so </wild speculation>, since it would not depend on a special position in an array.

 Quote:
Having all these 'spring' from the Prof.xml 'trunk' would be a way to illustarate/externalise them into an editable format. Right?

Is a 'spring' just a link or a reference to another file?
Editable format? Is the prof.xml file not editable?

 Quote:
lesh mentioned the most important case, the character's "role", or, better, what one can do with a character on a specific slot.

Is a role linked to a specific slot? Do i misunderstand the concept of a role?
Offline - OFFLINE
Top
Badges:
#181635 - 15 April, 2008 04:38 PM Re: prof.dat -> xml [Re: BirdFlu]
Khor1255
General
General

Posts: 5954
Loc: Pleasantville, NJ
What I meant by spring is that perhaps the prof.xml be the 'parent' file and the other proposed .xmls be referenced from there similar to how the items.xml is the parent file and so on.

Forgive me for lack of technical terminology.
_________________________
Dan Watson
Offline - OFFLINE
Top
#181694 - 16 April, 2008 06:19 AM Re: prof.dat -> xml [Re: Khor1255]
the scorpion
General
General

Posts: 5492
Loc: CH

______
BANNED
your last question birdflu:

yes, the some of the roles lesh mentiones are directly dependant on the character's slot.

there are many things, like factions like nationality like gender etc. that are actually defined by prof.dat, for many other things, very important things, prof.dat is just the "messenger" of the data or simply sais nothing about them.

e.g. slot 131 can't be at AIM, can't be an RPC, can't be a trader, can't be placed according to a script like character 68 or 61 (and tons of others) etc.

Let's think of the most important distinction: playable character vs. not-playable character

hardcoded. Do we really want to go through all the trouble externalising prof.dat without having the possibility to select for any given slot if the character can be a PC or just an NPC?

Wouldn't it even be necessary to have this first before we can add any slots to a new, open-ended prof.dat? wouldn't we require to hardcode the role of any additional character if we just externalise prof.dat and not the exe's slot restrictons?

a new character would have to have the possibility to be defined in prof.dat to appear at AIM, MERC, as an RPC, as an EPC as a Trader a vehicle a heli pilot, etc see lesh's list. prof.dat alone wouldn't allow that yet --> need to hardcode.

all of this is stored "higher" than prof.dat in the exe, and many of this things have logical connectons to "lower" data like maps, face files, speech and subtitle files etc.

I was hoping that we can tackle the entire complex of things by its head, which i, personally woudl say, is the slot assignment.

then we could go "down" subsequently to prof.dat, to the faces files etc. I am well aware i am asking probably impossible things here, but what i hope is give future modmaking the best possibilities, and one of the most important limitations are the slot limitations.


i think you saw the problem with this @0 thing. What we hope for is of course to have some sort of control over what happens if we pick @0 for any given character. Or, at least, that we can disable the script from happening ideally from within prof.dat
so maybe another flag that says "uses exe placement script yes/ no"

(ideally, the fle woudl also give an index of the script which would point to "placementscripts.xml" where we coudl select. But this is far away dreaming, kaiden and dealtar have writting some things about scripting, maybe then this could become topical)

currently, this is possible in very limited ways only. It seems to work for some npc's, not for others, some work partly, and sometimes we even break prof.dat if doing too many changes to the @0 entries (yes, it can cause a blackscreen-ctd at the beginning of the game and other ugly things)

vengeance NK may still have had it in certain versions.


of course, the impact of this scripts is vastly bigger as long as all the slot hardcodings exist. If a modmaker can just select NPC 131 to take over tony's or devin's or skyrider's role, the placement routines of said characters would restrict modding much less.


the reason why i put so much emphasis on these things is because i don't consider them "spring" of prof.dat at all, i consider many of these things a requirement for prof.dat to work efficiently. I consider the slot restrictions to not be dependant on prof.dat, but the other way around. your externalisation of prof.dat would probably suffer from this "parental" limitations the way i understand it.

of course, my grasp of these things is very limited, so if it can be explained to me how to solve the problems, or, that these problems are not really problems once you guy(s) have done your magic, then the better.

i hope my point gets across, it is sometimes difficult for me to correctly formulate these ideas as they come from very specific modmaker perspectives.
Offline - OFFLINE
Top
#181745 - 16 April, 2008 12:05 PM Re: prof.dat -> xml [Re: the scorpion]
Khor1255
General
General

Posts: 5954
Loc: Pleasantville, NJ
To my understanding in an .xml file it is possible to link files by having a 'trunk' or maybe 'root' .xml with branches stemming off of it (think Items.xml as the trunk and weapons.xml as a branch that has magazines, ammotypes, etc. as branches of this).

It doesn't matter what the 'trunk' .xml is called but I would think for characters the intuative choice would be prof.xml. In fact, I think if all the links to character rendering were externalised it is likely that everything 'springs' from the 'trunk' .xml of prof.dat.

Maybe not.

That is not the point though.

The point is, we need a set of .xmls linked together through a common 'trunk' or 'parent' .xml (forgive me for speaking in 'technical' terms I have only passing aquaintence with).


But in order for this to work in a manner 1.13 modders are already familliar with the natural (and perhaps only) choice would be to have a set of .xmls each preforming their specific functions (be they exact character slot, map placement, npc 'script' [meaning of course the actions an npc takes) etc.).

This is what I meant by other .xmls 'springing' from the prof.xml.

Think Items.xml in relation to it's 'child' directories.
_________________________
Dan Watson
Offline - OFFLINE
Top
#181764 - 16 April, 2008 03:45 PM Re: prof.dat -> xml [Re: Khor1255]
BirdFlu
Lieutenant
Lieutenant

Posts: 598
Loc: Lampukistan
 Quote:
.. about slots ..

That is very unfortunate that we have slots. I would say to hell with them, but this would imply too many changes at one time.
So maybe we should start slow. Actually i am trying to implement my operator overloading idea. The result should be a storage of
merc profiles independent of their ID or slot. This implementation will still have the old mapping (slot_id defines merc's relations
and behaviour), so no real improvement here. But once this works we could try to find a better mapping.

Anyway, we need a way to identify a merc. Here are some possibilities:
1. static location in an array (-> the current way)
2. identification by mercs name or nickname. Problem: multiple mercs could have the same name or nickname
3. identification by arbitrary numeric id defined in a file (ex. 1234,5263,6362566, 2535)
4. identification by alphanumeric id . Similar to a name, but probably more unique (ex. BZWBHd/&"§G$DB7, BZDW§B-SDFMU"§-NNWU)
4. something else.

Lets speak about roles. You say they depend on slots (which they do), but you also say that we should get rid of slots. How should then a role be assigned to a merc?

For me a role is a set of behaviours. A behaviour could be for example
1. Movement behaviour : Moving only on streets, around a specific sector or town etc.
2. Trading behaviour : Buying or selling only a limited set of items/weapons.
3. Comment behaviour : Comment frequency, comment topics, etc.
4. Combat behaviour : Switching to burst-mode,
5. i suppose you can think of more stuff

So the role of Tony is defined by his movement and trading behaviours.
But can a merc have multiple roles at the same time? For the answer yes and no we might have to define behaviours differently.

The ideal case would be when you could define/design a merc by defining his roles that can change over time. The roles itself are defined
as a set of behaviours that could be varied by additional parameters. Behavours would be linked to some functions in the code, but they
would not depend on a specific merc. The merc's profile would be just another parameter to that function.
Offline - OFFLINE
Top
Badges:
#181766 - 16 April, 2008 03:54 PM Re: prof.dat -> xml [Re: BirdFlu]
BirdFlu
Lieutenant
Lieutenant

Posts: 598
Loc: Lampukistan
If you as a modder create a new merc, what file structure would you like better?

1. Having data of this merc in ONE file, maybe referencing some very basic data in different file(s)?
In the end you would have 'N+m' files, where 'N' is the number of mercs and 'm' is the number of the basic data files you referenced to from the merc files.

2. Or having the data to a specific "topic" of ALL mercs in one file. You would have 'N' files, where 'N' is the number of "topics".
Offline - OFFLINE
Top
Badges:
#181769 - 16 April, 2008 04:18 PM Re: prof.dat -> xml [Re: BirdFlu]
Khor1255
General
General

Posts: 5954
Loc: Pleasantville, NJ
Doing away with the slots just to call them something different is arbitrary and confusing to current modders. There is nothing wrong with having characters defined by their i.d. numbers as they currently are.

The problem is the limitations that are present in these slot i.d.s. This is very similar to the limitations that used to be present in the items slots which determined (for instance) that firearms could only occupy slots 1-69 (and even some of these slots were unavailable).

Once the items were properly externalised these slot limitations had much less signifigance and finally were irrelevant.

This is the kind of freedom we need.

I think the character .xmls are going to be more complex in that their game interaction may be somewhat buried or scattered throughout the code. Also, the data which would need to be externalised for each character .xml type (prof.xml, quests.xml or whatever) may contain many more or less intuative type lines.

This is a huge task.

But if you wanted to present everything - all required fields into one .xml perhaps that wouldn't be bad it is just that it seems smoother to externalise one aspect at a time.


But the point is, you should not seek to change or do away with character slots just to name them something different but rather you should be looking to remove the resrictions that one group of slots has compared to another. So we might make an rpc occupy slot 152 for instance where now another one of the endless Santos brothers is just wasting space.
_________________________
Dan Watson
Offline - OFFLINE
Top
#181883 - 17 April, 2008 09:25 AM Re: prof.dat -> xml [Re: Khor1255]
the scorpion
General
General

Posts: 5492
Loc: CH

______
BANNED
yes, the identification by slots isn't necessarily bad. i think if the restrictions on slots can be broken up, the slots themselves wouldn't be much of a problem.

so what would be needed is an intermediate file between the exe and prof.dat that defines which slot can hold what type of character, especially the distinction PC/NPC and then also in some more detail (what kind of pc, traders etc)

now in theory, that's easy, but i can see a bunch of issues coming up there, e.g. this file would need to be able to cause many "slot-dependant" things to change.
Offline - OFFLINE
Top
#181918 - 17 April, 2008 01:26 PM Re: prof.dat -> xml [Re: the scorpion]
BirdFlu
Lieutenant
Lieutenant

Posts: 598
Loc: Lampukistan
OK, lets clarify some things. We have N merc profiles and they are stored in a list. Every position or location in that list can be referred
to as a slot. The problem now is the"VIEW" on the merc profiles. This view defines that slot A to B belong to AIM members and C is a
merc with a special behaviour and D to E are the player's characters, and so on. You can define a range (A to B), because you know
that you have a list (that is immutable!).

So how would you add new mercs? Assuming there were no free slots in the list, you would have to replace existing mercs. But actually you
don't really replace that merc, because a slot identifies a POSITION in the list and not the merc that happens to be in this slot. A merc
is defined by his/her attributes and behaviour. Attributes are defined in the merc's profile but not his behaviour. It is statically
linked to a slot. So by replacing only the the attributes you only replace a part of a merc.

(For the sake of simplicity i define membership of for example AIM as behaviour, as all AIM members behave in a similar way (they let hire them etc.))

Imagine now a merc's behaviour would be linked to his profile, then the order of your mercs would not matter anymore. You also don't
have identify them by some hardcoded id or slot number, you actually cannot do it because you cannot rely on the number and
order of mercs in the data files. The number and order could change anytime between two games, not necessary in a game. I suppose you
want that kind of flexibility or something similar.

Now you have to identify mercs by another id, a name for example. Important is that it is defined in a file and not somewhere in the code.
The mapping from name to merc is done by the program and no statement was made about the actual storage. It could still be a simple
list (as it is now) or multiple lists or a tree (map) or any other complex data structure. With this flexible approach you lose the
concept of a slot (which is a good thing in my opinion). It is as if they never existed.
Offline - OFFLINE
Top
Badges:
#181930 - 17 April, 2008 04:18 PM Re: prof.dat -> xml [Re: BirdFlu]
Khor1255
General
General

Posts: 5954
Loc: Pleasantville, NJ
If that is the easier way to do it than by all means it is the way it should be.


However, when you come to things like the speech files and map references which are clearly linked to the profile number in prof.dat you would either be changing all of this to now use name references or still using the slot numbers in some capacity.

If the use of these slot numbers could be restricted to only these functions I guess it would be alright. It just seemed to me that the method Madd Mugsy used where he still kept the item numbers yet extrnalised relevant data to the point where these numbers only became useful in listing was the more intuative approach.

If, within the program, the game calls references mercs by name instead of by slot number this would be invisible to both gamers and modders so it is a perfectly acceptable change (provided this change does not break something which I have a very uneducated hunch that...).

The main thing is to remave the resrictions placed on slots. If while doing this you could find a way to increase the number of slots all the better.

What we need is:

First, the ability to use any slot for any type of character so we could increase the number of npc/rpcs for instance.
Second, some control over the hidden routines a character must adhere to (@0 map placement, quest signifigance, etc.)

And while you are 'in there poking around' you might find a way to at least uncover some of the mystries of the unknown fields in the .npc files or better yet come up with an NPCs.xml.

This is really great work you are doing and I hope that I do not sound argumentive. I have come to believe that the less you change the easier the end result usually so I think it is better to externalise as much as you can and then change rather than the other way around.

But being I am a non coder my thoughts on this may be somewhat speculative.
_________________________
Dan Watson
Offline - OFFLINE
Top
#181975 - 18 April, 2008 04:16 AM Re: prof.dat -> xml [Re: Khor1255]
the scorpion
General
General

Posts: 5492
Loc: CH

______
BANNED
i think especially the .npc files and several hardcoded routines all over the place in the .exe use references to slot numbers.

so if you remove this "identification of mercs by slot number" i guess a big number of quest triggers and maybe the entire npc interaction code would have to be adjusted.

so my equally uneducated guess is that it might be a lot more complex to get rid of these slots than it appears at first.
Offline - OFFLINE
Top
#182004 - 18 April, 2008 06:24 AM Re: prof.dat -> xml [Re: the scorpion]
SpaceViking
Major
Major

Posts: 2954
Loc: Rochester, Minnesota, USA
Just move the slot number to a field in the data and use that to find specific NPCs. Have the code check that all the ID#s are unique when the characters are read in.
_________________________
What's a Space Viking? viking
Offline - OFFLINE
Top
Badges:
#182131 - 18 April, 2008 03:53 PM Re: prof.dat -> xml [Re: SpaceViking]
BirdFlu
Lieutenant
Lieutenant

Posts: 598
Loc: Lampukistan
 Quote:
Just move the slot number to a field in the data and use that to find specific NPCs. Have the code check that all the ID#s are unique when the characters are read in.

Yes, but there are some places in the code where you iterate over a certain range of NPCIDs, like:
 Code:
void UpdateMercMercContractInfo()
{
  UINT8	ubCnt;
  SOLDIERTYPE				*pSoldier;
  for( ubCnt=BIFF; ubCnt<= BUBBA; ubCnt++ )
  {
    ...
    gMercProfiles[ ubCnt ].iMercMercContractLength = pSoldier->iTotalContractLength;

This would force me create a continuous sequence of IDs. And we would again have the problem that, for example M.E.R.C. members,
have the IDs 40 .. 50. I would rather have something like this
 Code:
  Iterator it = gMercProfiles.IterateGroup("M.E.R.C.");
  for( it.begin(); !it.end(); it++ )
  {
    ...
    gMercProfiles[ it() ].iMercMercContractLength = pSoldier->iTotalContractLength;

Offline - OFFLINE
Top
Badges:
#182132 - 18 April, 2008 04:17 PM Re: prof.dat -> xml [Re: BirdFlu]
Khor1255
General
General

Posts: 5954
Loc: Pleasantville, NJ
Maybe then you could create a 'branch' from the prof.dat that checked on a seperate list so the numbers could be consecutive in the sub list or branch but would only be referred to by their ubIndex number.

So in other words, maybe a MERC.xml that could have any placement in the prof.dat (or whatever the 'parent' directory would be called) but have a specific number in it's own .xml that would be pointed to in the 'parent' .xml.

Again, I am borrowing from the way it is handled in the Items.xml 'tree'.

But I see no reason why this would not work provided all the relevant fields were externalised.
_________________________
Dan Watson
Offline - OFFLINE
Top
#182136 - 18 April, 2008 04:56 PM Re: prof.dat -> xml [Re: Khor1255]
SpaceViking
Major
Major

Posts: 2954
Loc: Rochester, Minnesota, USA
Yeah, using iterators would be better. Say, did they do that for the new inventory code?
_________________________
What's a Space Viking? viking
Offline - OFFLINE
Top
Badges:
#182141 - 18 April, 2008 05:37 PM Re: prof.dat -> xml [Re: Khor1255]
BirdFlu
Lieutenant
Lieutenant

Posts: 598
Loc: Lampukistan
@ Khor1255 : what? i'm having a hard time to understand you.

 Quote:
Maybe then you could create a 'branch' from the prof.dat that checked on a seperate list so the numbers could be consecutive in the sub list or branch but would
only be referred to by their ubIndex number.

What numbers could be consecutive (and why)? What would be referred to by their index number?

 Quote:
So in other words, maybe a MERC.xml that could have any placement in the prof.dat (or whatever the 'parent' directory would be called) but have a
specific number in it's own .xml that would be pointed to in the 'parent' .xml.

MERC.xml, is that the data for ALL mercs or for only ONE merc? Placement in prof.dat means slot?

 Quote:
Again, I am borrowing from the way it is handled in the Items.xml 'tree'.

I understand the structure of the items.xml file, so i try to extrapolate that structure to our case.
If i understand you correctly you propose a file 'mercS.xml' that holds a list of ALL mercs. Every entry for a merc has some fields like,
pointer to prof.dat ("you can find that merc in slot 'xyz'"), a new 'merc_id'. Then there is another file that has additional merc data,
that for every merc entry has a field which says "this data belongs to the merc with id 'merc_id'".

Has my description something in common with your description?
Offline - OFFLINE
Top
Badges:
#182142 - 18 April, 2008 05:43 PM Re: prof.dat -> xml [Re: BirdFlu]
BirdFlu
Lieutenant
Lieutenant

Posts: 598
Loc: Lampukistan
 Quote:
Yeah, using iterators would be better. Say, did they do that for the new inventory code?

I'm not sure, i haven't seen any iterators. But since a haven't looked into inv code too deeply i may be totally wrong here.
Offline - OFFLINE
Top
Badges:
#182181 - 19 April, 2008 04:38 AM Re: prof.dat -> xml [Re: BirdFlu]
Khor1255
General
General

Posts: 5954
Loc: Pleasantville, NJ
I see where using the term Mercs.xml would have been confusing. Sorry about that. We were taliing about the hireable characters from MERC at the time.


What I meant by a 'Mercs.xml' is just a way to isolate parts of the regular prof.dat list into 'branches' that could be defined by their own characteristics.

For example, maybe we would have an AIM.xml that would cover all the characters in AIM but also be opened to any addition we might want to add to that branch or array.
We might have one for Merc, one for Rebels, one for Kingpin's goons, etc. Or maybe it would not have to be divided so many times. Looking at the code you would know this far better than me.

Anyway, if we did it like this their Prof.dat index number would only be relevant for their placement in the Parent list. The "branch' lists or subdirectories might be opened eventually to add as many of one type of character (AIM mercs for instance) the modder wanted.

In this way we would not have to change all of the times the game called for a merc by his Prof.dat index. You could leave these numbers alone for now. With subdirectories you might have AIM characters occupying Prof.dat slots 0-39 but also (because the program would be looking at their uiIndex number instead of the Prof.dat slot number only) you might have AIM characters occupying slots 151-155 as well. The way the game would read it would be by the uiIndex (or whatever you would call it in the .xml) so that Prof.dat number 151 would be equal to uiIndex 40 in the AIM.xml (because it is added to the end of the AIM list.


This is exactly how the items.xml does it with one exception. In the Items.xml slot 0 has to be a nada (or blank) item. I do not know if this would be problematic.

But if you could make factions and such referenced by Prof.dat first but then REALLY referenced by a 'Child' directory like is currently the case with items a lot of the restrictions of slots might become arbitrary through externalisation of each faction type.

I am speaking largely from intuition here but I think if the 0 = Barry slot is not an insurmountable barrier in the Prof dat (maybe by making the last number in Prof.dat act as the beginning/end of the array) we might be able to do all sorts of cool things possibly including adding more characters to the total list.

But even if we couldn't do that, just being able to add as many of whatever type of characters we wanted (factions, rpcs, whatever) within the existing list would be golden.




Edited by Khor1255 (19 April, 2008 05:51 AM)
Edit Reason: clarifacation
_________________________
Dan Watson
Offline - OFFLINE
Top
#182240 - 19 April, 2008 12:53 PM Re: prof.dat -> xml [Re: Khor1255]
BirdFlu
Lieutenant
Lieutenant

Posts: 598
Loc: Lampukistan
 Quote:
AIM characters occupying Prof.dat slots 0-39 but also (because the program would be looking at their uiIndex number instead of the Prof.dat slot number only)
you might have AIM characters occupying slots 151-155 as well.

I see a problem that will eventually come up, because there are interdependecies between mers. The arrays bBuddy[5], bHated[5],
bMercOpinion[75] (prof.dat slot numbers) in a merc's profile describe the relationship to other mercs. If we now start to happily
change order of mercs or add new mercs in-between, then we would have to adjust this values. If we would have to do it by hand, there
will definitely be some errors during the conversion process. So maybe we should reference other mercs by name and not by their slot
number in the prof.dat file. This would make us 'immune' to changes in the merc list.
Offline - OFFLINE
Top
Badges:
#182251 - 19 April, 2008 02:49 PM Re: prof.dat -> xml [Re: BirdFlu]
the scorpion
General
General

Posts: 5492
Loc: CH

______
BANNED
why would you have to change the order of mercs?

wouldn't that be the task of somebody modifying the files? the only thing that needs to be done is creating the possibility of a flexible handling of the slots.

there will be many other dependencies, so by no means should the slots be shuffled, just the possibility to use any slot as wanted would be required.
Offline - OFFLINE
Top
#182256 - 19 April, 2008 03:50 PM Re: prof.dat -> xml [Re: the scorpion]
BirdFlu
Lieutenant
Lieutenant

Posts: 598
Loc: Lampukistan
 Quote:
why would you have to change the order of mercs?

Because. If it is possible then someone will do it. The game will (probably) not crash, but there will be some strange results (i.e. unfitting comments).

 Quote:
wouldn't that be the task of somebody modifying the files?

Exactly. And why should anyone forget to do it, it's so obvious. %-(

 Quote:
the only thing that needs to be done is creating the possibility of a flexible handling of the slots.
there will be many other dependencies, so by no means should the slots be shuffled, just the possibility to use any slot as wanted would be required.

If you add a new AIM merc right after the other AIM mercs (because it's nices to have them all together), all following mercs
will have another id/slot (+1). So if a merc in slot 55 likes another merc in slot 66, then after the insertion the number has to
change to 67 (or not, depending on where the new merc was inserted).

If the implementation can/will easily handle such cases, then everything is fine. If not, then we will have to do it in the files.
I'm just saying that this problem could occur and that we should be aware of it.
Offline - OFFLINE
Top
Badges:
#182268 - 19 April, 2008 06:48 PM Re: prof.dat -> xml [Re: BirdFlu]
Khor1255
General
General

Posts: 5954
Loc: Pleasantville, NJ
Yes, errors in data entry and use of these .xmls are indeed possible and will definately happen. Give a monkey any tool and he is bound to misuse it from time to time.

I have been guilty of incorrect data entry when working on my items.xmls (which are close to completely different from the vanilla Ja2 items) more than a few times.

But I would not trade this flexible and extremely versatile system for anything mod wise.

We are not asking for an error proofreading system. We NEED flexibility in character assignment.

It is actually quite easy to focus on all the parameters of an individual merc as you are working on him. If - for instance - you are working on Fox you probably know that she dislikes Buns so you adjust your new character accordingly. If you don't already know this, it is clearly written in one line of the prof.dat so you would be remiss to skip this line.

Of course, it gets a little more important when dealing with rpcs. But the concept is completely the same.

What we need in addition to versatile and flexible character .xmls are clearly written headers like what used to be supplied with the items .xmls. Right in these headers you could write the cautionary information you wish to convey like changing this field may result in unpredictable character interaction if the other character is not adjusted properly and changing that field may result in quest irregularity or disabling of a quest detail unless some other field is changed etc.

Again, we are not asking for a fully automated character editor just yet. We need the tools that will have to be developed to make this editor.

As I said before, I am not even concerned whether such a fully automated editor ever gets made especially if it must alter the base .xmls like they did to make the XML Editor for the items. When they made that editor certain changes were made to the xmls that made editing them manually a real pain in the ass. Now if the XML Editor could do everything manual editing could I would maybe be ready to make the change but it does not.

I can't repreat enough times that the kind of people who will be using the character xmls in depth are not you regular player/occasional tinkerer but rather those of us who likely do more modding than actual playing. We will make mistakes - yes - but we do not need and do not want to wait for a program that fully automates or makes this process idiot proof.

I think it can also be said that during our manual tinkering with the items .xml we gave Mugsy several ideas for what fields to externalise that may not have occoured to him right away. So this process - even the mistakes - was helpful to the finished product.








EDIT




As far as adding an AIM merc right after the last AIM merc in the prof.dat list this would - of course - be the incorrect way to do it. One of the points in having 'branch' directories is so that the prof.dat list could be somewhat ammended by the subdirecory list.

Take ammo in the items list for example, this normally started at item slot number 71 and ended at maybe slot 130 but in the magazines .xml it started at 0 and ended at 58 if I remember correctly. If you wanted to add another magazine you only had to find an empty item slot, add the magazine there and give it a uiIndex of 59 in the magazines xmls (after entering all the necessary data of course).

The same way could be done in whatever the parent .xml is for characters. I am beginning to feel as if it may not be the prof.dat because prof.dat starts with a 0 entry but this is a wild guess.

Whatever the main directory or parent file ends up being called, the most versitile (and already familliar) way to do this seems to be to link all various aspects or even factional differences to subdirectories so that when these lists are added to the parent .xml can still remain in tact with all it's dependant data functioning correctly.


Edited by Khor1255 (19 April, 2008 07:01 PM)
Edit Reason: addressing Bird Flu's last point
_________________________
Dan Watson
Offline - OFFLINE
Top
#182324 - 20 April, 2008 04:07 PM Re: prof.dat -> xml [Re: Khor1255]
BirdFlu
Lieutenant
Lieutenant

Posts: 598
Loc: Lampukistan
Well, i thought about the "It's your problem"-approach and i'm not sure if it will work out. I think we should take one extra moment and
create/design a 'better' system. The key is to make the the system understandable and maintainable. If you make a mistake and something goes
wrong you shoud be able to find out why this error happened and how to fix it as fast as possible (that is not searching for an hour only
to find out that you forgot a comma or so).

I would say the best approach is when a modification of any "game element" has to be as local as possible. So if i add a new merc i want
to only adjust a couple of lines in one file, and i don't want to make changes in a dozen of files, where a change in a file requires a
modification of multiple lines scattered over that file.

Anyway, here is a little update on the progress of my work.

This is the interface of the datastructure that should replace the MERCPROFILESTRUCT array:
 Code:
template<typename ValueType, typename IDType=int, typename NameType=wide_string, typename AllocatorType=DefaultAllocator<ValueType> >
class TDataContainer
{
	typedef TDataContainer<ValueType,IDType,NameType,AllocatorType> tCONTAINERTYPE;
public:
	typedef IDType		id_type;
	typedef NameType	name_type;
	typedef unsigned int	datalocation_type;

	const id_type invalid_id;

	/********************************************************************************************
	 *   Inner class 'Group' -> TDataContainer<>::Group
	 **********************/
	class Group
	{
		friend class tCONTAINERTYPE;
		typedef std::set<typename tCONTAINERTYPE::datalocation_type> tGROUP;
	public:
		/**************************************************************
		 *   inner class 'Iterator' -> TDataContainer<>::Group::Iterator
		 *************************/
		class Iterator
		{
			friend class tCONTAINERTYPE;
			friend class tCONTAINERTYPE::Group;
		public:
			~Iterator();
			void begin();
			bool end();
			// prefix
			void operator++();
			// postfix
			void operator++(int);
			/***********************************/
			ValueType& operator()();
			id_type id();
			name_type name();

		private:
			Iterator( typename tCONTAINERTYPE::Group& rGroup );
		private:
			typename tCONTAINERTYPE::Group&			_group;
			typename tCONTAINERTYPE::Group::tGROUP::iterator _current;
		};
		/*
		**************************************************************/
	public:
		Iterator GetIterator();
		/**
		 *   insert element
		 */
		bool AddElement(id_type element);
		bool AddElement(name_type element);
		/**
		 *   remove element
		 */
		bool RemoveElement(id_type element);
		bool RemoveElement(name_type element);
	private:
		Group(tCONTAINERTYPE& rCont);
	private:
		tGROUP			_groupelements;
		tCONTAINERTYPE&		_container;
	};
	/*
	********************************************************************************************/

	/********************************************************************************************
	 *   inner class 'DataElement'
	 ****************************/ 
	class DataElement
	{
		friend class tCONTAINERTYPE;
		friend class tCONTAINERTYPE::Group;
	public:
		DataElement( DataElement const& copy);
		~DataElement();

		ValueType& operator()()
		id_type id()
		name_type name()
		/***********************************************/
		bool RegisterID(typename tCONTAINERTYPE::id_type iID)
		bool RegisterName(typename tCONTAINERTYPE::name_type sName)
		bool RegisterGroup(typename tCONTAINERTYPE::name_type sName)

	private: // methods
		DataElement(	ValueType& value, id_type const& id, name_type const& name,
						typename tCONTAINERTYPE::datalocation_type location, 
						tCONTAINERTYPE &rCont);
	private: // data
		ValueType&	_value;
		id_type		_id;
		name_type	_name;
		typename tCONTAINERTYPE::datalocation_type	_location;
		typename tCONTAINERTYPE&			_container;
	} ;
	/*
	********************************************************************************************/
public:
	TDataContainer();
	~TDataContainer();

	void Clear();

	DataElement* NewElement(bool zero_memory=true);

	/******************************************/
	ValueType& operator[](id_type iID)
	ValueType& operator[](std::string const& sName)
	ValueType& operator[](WideString const& sName)
	ValueType& operator[](name_type const& sName)

	/******************************************/
	name_type GetNameForID(id_type iID)
	id_type GetIdForName(name_type sName)

	/******************************************/
	Group& GetGroup(name_type sGroupName)
	typename Group::Iterator IterateGroup(name_type sGroupName)

private:
	typedef std::vector<DataElement*>			tDATAELEMENTS;
	typedef std::map<name_type, datalocation_type>		tREGISTEREDNAMES;
	typedef std::map<id_type  , datalocation_type>		tREGISTEREDIDS;
	typedef std::map<name_type, Group*>			tREGISTEREDGROUPS;

	AllocatorType		_allocator;
	ValueType*		_invalid_element;
	tDATAELEMENTS		_merc_data;
	tREGISTEREDNAMES	_name_to_location_map;
	tREGISTEREDIDS		_id_to_location_map;
	tREGISTEREDGROUPS	_group_map;
};

I won't go into details, instead i will show hot it can be used.
We start with the declaration.
 Code:
// MERCPROFILESTRUCT gMercProfiles[NUM_PROFILES];
// MERCPROFILEGEAR   gMercProfileGear[NUM_PROFILES];
TDataContainer<MERCPROFILESTRUCT> gMercProfiles;
TDataContainer<MERCPROFILEGEAR>   gMercProfileGear;

typedef TDataContainer<MERCPROFILESTRUCT,int,wide_string,BlockAllocator<MERCPROFILESTRUCT,32> > MERCPROFILES;
MERCPROFILES gMoreMercProfiles;


Then we have to fill this container somehow
 Code:
while('read merc data from file')
{
  MERCPROFILES::DataElement *el = gMercProfiles.NewElement();
  el->RegisterID( ID );
  el->RegisterName( "name" );
  el->RegisterGroup( "group_name" );

  MERCPROFILESTRUCT &mps = (*el)();
  FillProfileWithDataFromFile( &mps );
}

The main steps are :
1. Create new element with 'NewElement()'
2. Assign a numeric and/or alphanumeric ID to this element
3. Fill fields of the contained data structure in 'DataElement'

Once the container is filled we can use it in many ways
 Code:
MERCPROFILESTRUCT &mps =  gMercProfiles[id]; // same syntax as now -> requires no additional changes in the code

gMercProfile.GetGroup("AIM").AddElement(gMercProfiles.GetIdForName("Bubba"));
MERCPROFILES::Group::Iterator it = gMercProfiles.IterateGroup("AIM");
for(it.begin(); !it.end(); ++it)
{
  cout << "merc's registered name : " << it.name() << endl;
  cout << "merc's real name       : " << it().zName << endl;
}


Since the container class is generic, it should also work with items (INVTYPE) or other data structures.

Right now, i am using this container class for 'MERCPROFILESTRUCT's and 'MERCPROFILEGEAR's and i had to change only the files
'Soldier Profile.*' and 'XML_StartingGear.xml' (and some minor modifications in other files).
I can start new games (old and new inv.) and i can save and load savegames without any changes in other files.
Offline - OFFLINE
Top
Badges:
#182469 - 22 April, 2008 06:58 AM Re: prof.dat -> xml [Re: BirdFlu]
Kaerar

General
General

Posts: 6343
Loc: Australia :D

______
Slideways through life
Original 1.13 SCI Creator *Updated*
I'm with Birdflu on this. Its not about making something entirely new, its about fixing an old limitation. If mercs were referenced by name rather than slot number it allows for much more flexibility and ease of use.
_________________________
"The bank hath benefit of interest on all moneys which it creates out of nothing."
Sir William Paterson - Founder of the Bank of England
Replacement UI Project Beta
Offline - OFFLINE
Top
#182480 - 22 April, 2008 09:10 AM Re: prof.dat -> xml [Re: Kaerar]
the scorpion
General
General

Posts: 5492
Loc: CH

______
BANNED
in what respect is a name easier to use than a slot number??

especially in modmaking, where mercnames might change hourly?

personally i find in the trader inventory xml section, which is pretty much the only place where actual names were used, it's a freakin mess.

but i guess if an external name changes, all dependant data changes too? what about trader inventory xml names (lol, just kidding)

i guess i don't understand the innovation in using a character's name for identification over a number. The number remains the same at all times while NPC/ PC names migth change hourly.

i'd probably prefer a number i think.
Offline - OFFLINE
Top
#182488 - 22 April, 2008 10:00 AM Re: prof.dat -> xml [Re: the scorpion]
Marlboro Man

Marshall
Marshall

Posts: 7525
Loc: USA
I agree with scorpion here. (gasp) \:\)
_________________________
DAG NAB IT !
Offline - OFFLINE
Top
Badges:
#182513 - 22 April, 2008 12:02 PM Re: prof.dat -> xml [Re: Marlboro Man]
BirdFlu
Lieutenant
Lieutenant

Posts: 598
Loc: Lampukistan
 Quote:
in what respect is a name easier to use than a slot number??
especially in modmaking, where mercnames might change hourly?

Because the registered 'name' does NOT change hourly. How that? The merc's name is just one of many attributes, that can change hourly.
The registered name, however, is assigned to the merc's data object. It is not the ingame name of the merc.
Example:
1. JA2::Larry::Normal
2. JA2::Larry::Stoned
3. RenegadeRepublic::Merc1::MercStatus
4. RenegadeRepublic::Merc2::MercStatus

You have to distinguish case 1 and 2, because they have the same ingame name (there are also two Joe's).

What we need is a unique ID for every merc for every situation. I proposed a numeric and/or alphanumeric ID. What we take in the end is one of the points in this discussion.

Imagine that in the future we could add (and remove) mercs really easy so that we could have Merc Packs. How do you ensure that the IDs
of mercs from different packs don't overlap AND the newbie user doesn't have to make adjustments.

With names you could distinguish mercs by prepending the mod's name for example : ModName::Version::Merc1::Status, ModMakerName::MercName::Status, etc.

With numbers you have to be sure that nobody takes the ID 171, because you already did. It is possible and it is not even that hard, but it is still somewhat risky.
Offline - OFFLINE
Top
Badges:
#182555 - 22 April, 2008 04:59 PM Re: prof.dat -> xml [Re: BirdFlu]
BirdFlu
Lieutenant
Lieutenant

Posts: 598
Loc: Lampukistan
 Quote:
Its not about making something entirely new

Actually there is something new, a conceptually new approach to merc enumeration or iteration.
The merc's profiles are stored in a plain C-array. We work on these profiles by
1. directly accessing the array with a hardcoded ID respectively array position (the infamous slots)
-> the restriction is that there are hardcoded behaviours and events linked to a special ID and thus to the corresponding slot.
So by replacing that merc we not only use his slot, we also trigger his events, which is most probably not desired.

2. iterating over a subrange of all mercs, for example AIM mercs (ids 0 - 39)
-> the restriction is that the subranges for groups of mercs are hardcoded and there are no free slots between the groups (would be a waste of
memory, because the array would have have MAX_ID slots with less than MAX_ID occupied slots)

The new concept is that we now directly iterate over groups of mercs. Therefore we are not dependent on any predefined slots.
The implementation (see above) allows us to use both concepts, but as long as there is one place in the code where we use hardcoded
ids, we cannot abandon them and thus we gain very little flexibility. BUT, being able to use both concepts, we can tranform and test the
code file by file and function by function. I think this is a great advantage.

Speaking about the code, has anyone looked at the class interface above, especially other coders. Because if we want some progress
we have to start transforming the code. And since this changes would affect larger parts of the code, you would have to use this
data structures eventually. So if you don't like something you should say it now.
(unless, of course, we discussed for amusement only )
Offline - OFFLINE
Top
Badges:
#182587 - 23 April, 2008 05:17 AM Re: prof.dat -> xml [Re: BirdFlu]
the scorpion
General
General

Posts: 5492
Loc: CH

______
BANNED
[quote=BirdFlu]
 Quote:
So by replacing that merc we not only use his slot, we also trigger his events, which is most probably not desired.


the tricky thing will be that sometimes, this might be desired. The point of flexibility would be to be able to decide about that externally.

see a modmaker might deliberately place his new merc on slot (say) 61 to have a special placement routine, use the existing script and files and the trader interface

and have, in exchange, the merc currently occupying slot 61 moved to a diffderent slot, say slot 120 something.

using existing functions by way of overwriting can be a very powerful tool, and the slots as they are right now make it easy to use this, the limitations here are where new behaviour is wanted, especially making former NPC slots "hireable" or add new NPC's, traders and hireables.
Offline - OFFLINE
Top
#182594 - 23 April, 2008 05:56 AM Re: prof.dat -> xml [Re: the scorpion]
Kaerar

General
General

Posts: 6343
Loc: Australia :D

______
Slideways through life
Original 1.13 SCI Creator *Updated*
I can't see how this would break that behaviour. All it would mean is that the Slot would be use a slightly different reference value (the name or alphanumeric ID if ya like) instead of slot 61. You can still copy to slot 120 without a problem. You could even duplicate characters this way rather than just have the ability to overwrite.

Also if the link is the name if that gets tracked through any linked XML and auto changed in the editor, that would make life a lot easier.
_________________________
"The bank hath benefit of interest on all moneys which it creates out of nothing."
Sir William Paterson - Founder of the Bank of England
Replacement UI Project Beta
Offline - OFFLINE
Top
#182641 - 23 April, 2008 11:11 AM Re: prof.dat -> xml [Re: Kaerar]
Khor1255
General
General

Posts: 5954
Loc: Pleasantville, NJ
I still do not see any good reason to do away with the slot NUMBERS.

All we want is slot limitations being removed or at least externalised.

If this were approached like the items .xmls (with subdirectories cross referenced according to factional/quest/etc) we could neatly work on each character in a way consistant with what we already know how to do.

But if you decide on another way and this way actually works I can only say that I will definately give it a try.

I see zero advantage in just arbitrarily doing away with slot numbers especially when so much dependant data is referenced by these numbers.
_________________________
Dan Watson
Offline - OFFLINE
Top
Page 1 of 3 1 2 3 >


Moderator:  Dieter, lockie, Marlboro Man, Tbird94lx 

Forum Stats
7993 Members
52 Forums
7842 Topics
159590 Posts

Max Online: 228 @ 21 September, 2007 02:08 PM
Copyright 2000 Bear's Pit