r/learnjava • • 8d ago

Building a custom Java data structure library for my game, would like feedback.

I recently released JArray, a Java data structure library I've been working on.

The goal was to build something that combines dynamic sizing, specialized primitive storage, mixed data types, and multidimensional array functionality because I thought it might help me with my game-development, things like inventory, maps, and parsing save files.

it currently uses a whole bunch of primitive arrays, and metadata array to hold values. but i don't know if its better to use a single object array.

I'd appreciate feedback on the API design, storage architecture, and features that would make the library more useful, also any questions about the library.

here's the GitHub to the project. hopefully it doesn't get flagged as promotion: https://github.com/AJ-1-code/JArray

2 Upvotes

26 comments sorted by

•

u/AutoModerator 8d ago

Please ensure that:

  • Your code is properly formatted as code block - see the sidebar (About on mobile) for instructions
  • You include any and all error messages in full - best also formatted as code block
  • You ask clear questions
  • You demonstrate effort in solving your question/problem - plain posting your assignments is forbidden (and such posts will be removed) as is asking for or giving solutions.

If any of the above points is not met, your post can and will be removed without further warning.

Code is to be formatted as code block (old reddit/markdown editor: empty line before the code, each code line indented by 4 spaces, new reddit: https://i.imgur.com/EJ7tqek.png) or linked via an external code hoster, like pastebin.com, github gist, github, bitbucket, gitlab, etc.

Please, do not use triple backticks (```) as they will only render properly on new reddit, not on old reddit.

Code blocks look like this:

public class HelloWorld {

    public static void main(String[] args) {
        System.out.println("Hello World!");
    }
}

You do not need to repost unless your post has been removed by a moderator. Just use the edit function of reddit to make sure your post complies with the above.

If your post has remained in violation of these rules for a prolonged period of time (at least an hour), a moderator may remove it at their discretion. In this case, they will comment with an explanation on why it has been removed, and you will be required to resubmit the entire post following the proper procedures.

To potential helpers

Please, do not help if any of the above points are not met, rather report the post. We are trying to improve the quality of posts here. In helping people who can't be bothered to comply with the above points, you are doing the community a disservice.

I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.

9

u/aqua_regis 8d ago

Not to be mean, but your entire library is code smell, alone the idea of your "data structure" is code smell and indicates a lack of planning beforehand.

You are not gaining anything with your data structure. Your data structure has more overhead than even the wrapping of primitives would cause (Java has autoboxing and autounboxing).

The need to store different, unrelated data types in a single data structure is smelly.

There are way too many static methods.

0

u/fishybusinessdev 3d ago

Lack of planning? No. I planned when I built this. And built it for exactly what I wanted it to do. Nd array structure without pointer chasing or boxing overhead. Which was (and still is) for my game, and works just fine. It’s definitely not for everyone. And it does have some overhead. But not that much. And what do you mean too many static methods? How can one have too many static methods? 

3

u/voidiciant 8d ago

hmmm….it’s clearly missing documentation on the api. Just by looking at it, why do I have getters that take a single index and others take a list of indices, what is “Object get(int... indices)” supposed to mean?

What is the semantic of a dimension “0”?

Then, I’m not sure if I like loosing my typeinformation for my own Objects, as you can only ever return Object and I need to keep track of casting, or am I missing something (or am I too rusty).

1

u/fishybusinessdev 3d ago

First off, thanks for the feedback I’ll make sure to include more details in the read me. But to answer your questions: since jarray can have dimensions like int[][][]. The get method uses Varags so you can do .get(1, 2, 3) which is the same as int[1][2][3]. Second the 0 is the initial capacity. So when you create a jarray with no params it’s initial capacity is 0 and 1 param is the set initial capacity like in array list. Since it grows dynamically. Finally yes you do(for now) need to keep track of casting. But this is the first release and I’m working on making it more user friendly.

2

u/Ormek_II 8d ago

I like the read.me. You spend some time explaining what you are doing and you did actually benchmark and try to be complete. Well done.

You write: “The important takeaway is that JArray can offer substantial advantages for certain workloads, especially when working with homogeneous primitive data.” 🤔 if the data is homogeneous, shouldn’t you compare with Array instead of ArrayList?

What we all miss: WHY? What is it, that you wanted to do, that you needed that abstraction for?

2

u/fishybusinessdev 3d ago

For your first question, yeah, comparing against a raw  int[]  makes total sense if the data is homogeneous AND fixed-size. The issue is when the size isn't fixed: in Java, you get forced into  ArrayList<Integer>  and pay the boxing tax, which I wanted to avoid by keeping values in primitive backing arrays. On top of that,  JArray  supports arbitrary N-dimensional shapes. While Java lets you do  int[][][] , the way Java stores jagged multi-dimensional arrays forces pointer chasing across memory. Because jarray flattens everything into a single 1D primitive array under the hood, it avoids that pointer chasing. As for your second question the "why" it actually started as a helper class in a game I'm making. I used it to store the multidimensional map efficiently and to easily slice out chunks of the map, rotate them, and manipulate them without writing messy index math everywhere. Eventually, I decided to turn it into a full standalone library just as a fun project and share it in case anyone else finds it useful. I hope this answers your question. 😄

2

u/Ormek_II 3d ago

It does.

I especially like the WHY: you wanted to „natively“ support your game map requirements in the data structure directly and efficiently. Makes sense; who am I to judge.

1

u/Ormek_II 8d ago

Especially in gaming: why all those features of python: reshaping, slicing, …

3

u/vegan_antitheist 8d ago

You clone an array by copying each value. You should first learn how computers use memory. The JVM can't always optimise your code.

1

u/istarian 7d ago edited 7d ago

That's pretty much how it works when your current array is too small and you need a bigger one. You make a larger array and copy over the data.

What are you suggesting as an alternative?

2

u/vegan_antitheist 7d ago

Yes, it's not about copying arrays, it's about how you do it. Doesn't java even have a clone method on arrays? And to copy to a larger array you can just use the Arrays.copyOf method. The JRE might optimise your code, but this is guaranteed to be optimal. And it's easier to read too.

1

u/fishybusinessdev 3d ago

Yeah. And I did try using that but wasn’t impressed with the full results. And I actually do use bulk range copy for contiguous data. The element by element copy is for arrays with non contiguous data like transposed arrays. I actually borrowed this design choice from numpy. I hope this makes things clear. 

1

u/vegan_antitheist 3d ago edited 3d ago

The Iterable passed to snapshot is never just a Collection that can clone itself? I guess you wouldn't call that method then. You don't even use Arrays.fill to fill the rest of the values in JArray.filled.

1

u/fishybusinessdev 2d ago

I’m like 90% sure javas collection interface doesn’t have a clone method. As for the filled method. I probably could use arrays.fill for some of it, but my loop and add method use the type detecting and sorting that makes the code clean.

2

u/vegan_antitheist 2d ago

And Collection has toArray. It might just use the default method of the interface but that would do the same as your method.

1

u/fishybusinessdev 2d ago edited 2d ago

yep, i forgot about that. as for the mutable part? thats just a design choice. i picked mutable. your code looks good, although wouldnt there be a bug in the code with set? since the actual size is still 0? also, wouldnt there be inconsistencies in mutability, and null handling? since ArrayList and CopyOnWriteArrayList return mutable lists, with null elements. LinkedList, and Vector return non-mutable lists. finally isnt the I extends Iterable redundant? it doesnt appear to be used anywhere. im guessing this isnt a full implementation, just a pseudo code example? and thanks for all the comments, its really helping me see parts of my code that need to be updated. (accidentally replied to the wrong comments, supposed to be replying to youre most recent reply)

1

u/vegan_antitheist 2d ago

It sucks that we still have to use the marker interface Cloneable but in this vase it works just fine. An ArrayList can clone itself internally. What's really stupid is that Java doesn't use anything to make immutable collections, so you end up with unnecessary clones in such a generic method.

1

u/fishybusinessdev 2d ago

true, Do you think using instanceof to detect ArrayLists and other Collections might optimize this, or would it be better practice to keep the generic iteration?

Something like:

private static List<Object> snapshot(Iterable<?> values) {

    if (values instanceof ArrayList<?> arrayList) {
        return new ArrayList<>(arrayList);
    }

    if (values instanceof Collection<?> collection) {
        return new ArrayList<>(collection);
    }

    List<Object> copy = new ArrayList<>();

    for (Object value : values) {
        copy.add(value);
    }

    return copy;
}

I'm not sure whether this would actually improve performance, or if it would help with avoiding unnecessary copies of immutable collections, though. i can look into it though.

2

u/vegan_antitheist 2d ago

Actually, checking if they are marked by Cloneable doesn't work at all. I forgot how much Java sucks when it comes to cloning things. They didn't have default methods and annotations came even later. It seems weird that they still haven't fixed this, but it's a technical debt they can't just fix now. So Objec.clone isn't optimal. It really depends on what you need. Should the list be mutable? But why would a snapshot be mutable?

Here are some optimisations I can think of:

static <E, I extends Iterable<E>> List<E> copyOf(I iterable) {
  if (iterable instanceof Collection<?> collection) {
    return switch (collection) {
      // ArrayLists are commonly used in Java
      case ArrayList<?> a -> (List<E>) a.clone();
      // Linked lists suck, so this returns an immutable list instead
      case LinkedList<?> l -> (List<E>) List.copyOf(l);
      // That's even worse than LinkedList
      case Vector<?> c -> (List<E>) List.copyOf(c);
      // This is another one that is used
      case CopyOnWriteArrayList<?> c -> (List<E>) c.clone();
      // this can be copied without an iterator:
      case List<?> list when list instanceof RandomAccess -> {
        int srcSize = collection.size();
        List<E> dest = new ArrayList<>(srcSize);
        for (int i = 0; i < srcSize; i++)
          dest.set(i, (E) list.get(i));
        yield dest;
      }
      // The collection might be able to optimise how it creates the array.
      default -> (List<E>) Arrays.asList(collection.toArray());
    };
  }
  // Any other iterable can be cloned like this:
  List<E> dest = new ArrayList<>();
    Iterator<? extends E> si=iterable.iterator();
    while (si.hasNext()) {
      dest.add(si.next());
    }
    return dest;
}

1

u/fishybusinessdev 3d ago

sorry i havent replied to anyones comments, but thank you all for youre feedback, it really helps, and i have been working on fixing, and optimizing the code, and figuring out where jarray fits into my java programming. ill let you all know once the newest version is released.

5

u/gekigangerii 8d ago

Seems like this is fighting against the idea of declaring your types upfront,

but ultimately you need to know what data type you stored because you have to handle it at compile time

1

u/Fuzzy-System8568 6d ago

Ill be honest, the benchmarks are nonsense. E.g I was interested in your multi-type benchmark? You add them all to the array and then check its size...

No calling the elements to use, no actually using any of the functionality of your array. Just... checking its size..there is no validation to check the correct data is in the right place, how fast retrieving an element of the array is... nothing.

That alone had me questioning the credibility of the library...

1

u/fishybusinessdev 3d ago

Yeah. The benchmarks aren’t the best. But I’ve been working on a jmh test. And I will be fixing the actual test cases. Thank you for you’re feedback.

1

u/TheGratitudeBot 3d ago

What a wonderful comment. :) Your gratitude puts you on our list for the most grateful users this week on Reddit! You can view the full list on r/TheGratitudeBot.

2

u/Available_Safe1818 8d ago edited 8d ago

Can you show more examples, like where it's better than an arraylist?

Why do you have all of the static final ints in Storage.java that mirror the enum values in DataType.java?

10 arrays inside of each instance of Storage seems like a strange choice.